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Follow-up Study of Schools Implementing Comprehensive School 

Reform 

Program Status 

Executive Summary 

Comprehensive School Reform (CSR) programs were implemented in 1998 in five states 
working with the Southwest Educational Development Laboratory (SEDL). SEDL’s five- 
state region includes Arkansas, Louisiana, New Mexico, Oklahoma, and Texas. This 
study examined the implementation status of CSR programs since fimding ended. Two 
huni"ed and sixty-eight surveys were distributed to schools that received CSR fimding. 

In addition, on-site visits were conducted to a sample of school in each state. School and 
student performance, as available was also examined. 

In 1997, Congress approved the Comprehensive School Reform Demonstration (CSRD) 
program, created to encourage schools nationwide to implement research-based, 
comprehensive, school-wide reform initiatives, especially in low-performing schools. 
With the passage of the 2001 Elementary and Secondary Education Act (ESEA), also 
known as the No Child Left Behind (NCLB) Act, CSRD is now known simply as 
Comprehensive School Reform or CSR (as it is no longer a “Demonstration” program). 
The program is regulated by the new CSR authority in Part F of ESEA’ S Title I and the 
Fund for the Improvement of Education. 

CSR schools in the states that SEDL serves have implemented a variety of reform models 
with 51 different models identified. The CSR program required schools to incorporate 
and address nine components related to school improvement. The goal of any CSR 
program is to produce long-term, lasting reforms in schools that result in sustained 
improvements in student performance. Assessing program continuation after start-up 
fimding ended is essential to understanding the factors that affect program continuation, 
including funding sources, program support, staff training, and model provider support. 

To assess the current status of each CSR program, survey questioimaires were developed 
and sent to all 268 CSRD-funded schools. The rate of return (approximately 40 percent) 
varied significantly by state from 23 percent in Texas to 75 percent in Arkansas. To 
supplement the mail surveys, site visits were conducted at fifteen schools. Only schools 
indicating a willingness to provide additional information and approved by the state 
education department were visited. School programs were selected to reflect different 
geographic locations, model types, school type, and student demographics. In addition to 
the survey and the on-site visits, selected published student performance data were 
examined. 

Of the 106 returned surveys, 79 were from schools still implementing the original model, 
whereas 27 schools had discontinued the original CSR model. Where the program is still 
in place, about 70 percent said the entire program is still in place while 24 percent said 
about half is in place. Four main reasons were given for the program remaining in place: 
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successful student outcomes, alignment with school goals, continued funding, and strong 
support by teachers and parents. Successful student outcomes were the most important 
reason noted. These reasons for program continuation were also confirmed in the site 
visit interviews. 

Site visit interviews indicated that many school have made a large investment in the CSR 
program. The comprehensive nature of many CSR models requires that a school make a 
large investment of time and money. Respondents in some schools said they lacked the 
desire or the energy to start a new program. Site visit interviews also revealed that 
program continuation is related to administrative and school staff support. In many 
instances, school principals were fully committed to ensuring continuation of the CSR 
model and school staff members felt that they had district support. Where programs were 
no longer in place, the primary reason centered on funding either in general or that the 
model was too expensive. Following in importance, respondents selected “did not have 
support by teachers or administrators,” and “chose another program.” 

During site visits, opinions among interviewees varied regarding the degree to which 
diminished funding has reduced the effectiveness of CSR models. Although most agreed 
that the models are expensive to maintain, they also indicated that the expenses 
associated with the models have decreased since the first years of implementation. Many 
principals question how they will be able to sustain the model over time. Without 
continued funding, program staff are concern that models will be modified to such an 
extent that their effectiveness will be diminished. 

Although funding was given most often as the reason for discontinuation, it is likely that 
the reason is more complicated. From the site visits and comments made on surveys, if 
the belief in the program was strong, so was the motivation (and success) to secure 
alternative funding sources. One of the major premises of any type of developmental 
program is building local capacity with a (presumed) reduction of direct (and expensive) 
support from the model provider. One component of the CSR program “Coordination of 
Resources” saw the largest difference between continuing and non-continuing programs 
with little coordination noted in the latter. 

Survey respondents were generally pleased with the level of support that they received 
from the model providers and said that proper training, particularly in the first two years 
of implementation, was critical to the model’s success. Fewer respondents where the 
program is no longer in place said that the quality of the model and support were 
excellent. In some cases, consultants provided informal help even after the CSR funding 
had ended. 

To be considered successful, CSR programs must improve student achievement. While 
some schools’ programs have not yet achieved expected results, respondents who choose 
to continue the CSR model believe the model will produce improved student 
achievement. Many school administrators said that first three years of implementation is 
not sufficient time to see marked changes in student performance. CSR programs are not 
limited to a particular model; nor are any one goal common in all models. Allowing 
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emphases to be placed where appropriate leads to a diversity of approaches and models. 
While this contributes to the strength of a general program such as CSR, it also 
complicates the issue of measuring student performance. 

Although the success of a reform model can be measured in a variety of ways_enhanced 
professional development, increased and improved instructional resources, and reduction 
in staff tumover_most success measures involve the student. Typical student outcomes 
include increased self-esteem, greater classroom participation, and higher test scores. The 
term “success” is dependent on what is trying to be accomplished by the program. 
Externally-defined success, such as graduation rates, may overlap with the internal 
measure, but are defined by the state. In some cases, this external definition of success 
may conflict with the internal goals of the program. 

A critical precursor for any effort to increase student performance is to have measurable 
goals and objectives so that progress may be ascertained in a concrete, impartial manner. 
In schools where the model continues, 83 percent of survey respondents indicated that 
they have developed measurable goals and benchmarks as compared to 67 percent in 
non-continued programs. 

Nearly all (98 percent) of respondents in schools where the CSR model is still in place 
noted a positive impact on student achievement. Even in sites where the model is no 
longer used, 88 percent reported a positive impact. Moreover, respondents identified a 
correlation between the quality of the model and whether the program was still being 
implemented in 2003. 

Data obtained fi-om all Texas schools that implemented a CSR program indicate that 
students in these schools made significant gains in performance fi-om 1998 to 1999; 
however, by 2001, those schools’ performance had dropped to approximately the same 
level as economically disadvantaged students in the state overall. (In Texas, the 
percentage of economically disadvantaged student is greater in CSR schools than in the 
state in general.) 

The range of educational reform models adopted by schools receiving CSR grants 
suggests that a variety of needs-and, therefore, a range of outcomes-were important to 
school staff. Accordingly, desired outcomes and approaches differed among schools 
implementing the same model, whereas similar outcomes were often within the goals of 
CSR schools implementing different models. The absence of a common goal or 
standardized measure limits this study’s capacity to generalize about the overall impact of 
the CSR program. Several schools implemented more than one model (usually two) 
operating concurrently. The degree of emphasis on each model has varied fi-om time to 
time, with one model taking precedence over the other as the importance of outcomes 
shifted. One school that implemented four programs at one time, now has none. 

All respondents were asked to describe the most important changes in their schools that 
resulted fi-om their CSR program. Several positive results were identified, both for 
schools with models still in place and for those no longer in place. None of the 
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respondents indicated that negative changes have occurred as a result of implementing 
the CSR program. Respondents of both survey types said that the CSR program had a 
positive impact on professional development. 

Respondents from continuing schools said that several positive changes have taken place 
as a result of the CSR program. Changes include improved student performance, valuable 
staff development, improved teaching strategies, greater availability of resources and 
materials, increased teacher and school staff morale, and increased parental and 
community involvement. 

Study Highlights 

© 106 of 288 surveys were returned. Of these, 79 were from schools that are still 
implementing the original model, 27 were from schools that had discontinued the 
original CSR model. In schools where the program is still in place, about 70 
percent said the entire program is still in place while 24 percent said about half is 
in place. 

© Four main reasons were given for the program remaining in place: successful 
student outcomes, alignment with school goals, continued funding, and strong 
support by teachers and parents. 

© The comprehensive nature of many CSR models requires that a school make a 
large investment of time and money. Because of this three-year large investment, 
many schools and staff were reluctant to discontinue the program. 

© The majority of respondents agreed that CSR programs are expensive to 

implement and maintain. Although most agreed that the models are expensive to 
maintain, they also indicated that the expenses associated with the models have 
decreased since the first years of implementation. 

© Though many programs continued to secure funding, several principals were 
unsure regarding future funding for their programs. Many principals question 
how they will be able to sustain the model over time. Without continued funding, 
program staff are concerned that models will be modified to such an extent that 
their effectiveness will be diminished. 

© Although funding was given most often as the reason for discontinuation, funding 
was found to be related to perceived program success. If the belief in the program 
was strong, so was the motivation (and success) to secure alternative funding 
sources. 

© Survey respondents were generally pleased with the level of support that they 
received from the model providers and said that proper training, particularly in the 
first two years of implementation, was critical to the model’s success. 
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© While some schools’ programs have not yet achieved expected student 

performance results, respondents who choose to continue the CSR model believe 
the model will produce improved student achievement. Many school 
administrators said that first three years of implementation are not sufficient time 
to see marked changes in student performance. 

® Program staff indicated several positive program outcomes: improved student 
performance, increased student and staff self-esteem, greater classroom 
participation, higher test scores, valuable staff development, improved teaching 
strategies, greater availability of resources and materials, increased teacher and 
school staff morale, and increased parental and community involvement.” 

® Nearly all (98 percent) of respondents in schools where the CSR model is still in 
place noted a positive impact on student achievement. For schools that 
discontinued the program, 88 percent reported a positive impact. 
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Follow-up Study of Schools Implementing Comprehensive School Reform 

Programs: 

Status Report 

Schools in Southwest Educational Development Laboratory’s (SEDL) five-state region 
implemented Comprehensive School Reform (CSR) programs in 1998. SEDL’s region includes 
Arkansas (26 sites), Louisiana (64 sites), New Mexico (21 sites), Oklahoma (30 sites), and Texas 
(137 sites). Not all of these 278 original sites continued their CSR programs through their three- 
year funding cycle. This study examines the following four aspects of the CSR programs; 

© Student performance 
0 Program components still in place 
0 Factors contributing to program continuation 
© Information on to the model provider 

Academic Information Management, Inc. developed this report using information obtained from 
a mail survey, school performance data, and site visits conducted at schools in all five states. 

Background 

The 1 993 RAND Corporation’s publication. Federal Policy Options for Improving the Education 
of Low Income Students, Volume I: Findings and Recommendations, suggested that the impact of 
the federal government’s Title I programs targeting schools with a majority of low income 
students would be greater if educators implemented schoolwide reform initiatives instead of ■ 
selective programs. Based on this and other research conducted in the late 1980s and early 
1990s, the Title I program, in the 1994 reauthorization of the Elementary and Secondary 
Education Act (ESEA), encouraged schoolwide initiatives among schools in which at least half 
of the students were disadvantaged. 

In 1997, Congress approved the Comprehensive School Reform Demonstration (CSRD) program 
to encourage schools nationwide to implement research-based, comprehensive, schoolwide 
reform initiatives, especially in low performing schools. The majority of schools were awarded 
approximately $50,000. The CSRD program required schools to incorporate and address nine 
components related to school improvement: (1) employing proven, scientifically based methods; 
(2) integrating comprehensive and aligned curricula; (3) developing measurable goals; (4) 
attaining and ensuring administrative, teacher, and parental support; (5) providing staff 
development; (6) including meaningful parental and community involvement; (7) using external 
technical assistance resources; (8) employing effective program evaluation techniques; and (9) 
producing increased student performance results. With the passage of the 2001 Elementary and 
Secondary Education Act (ESEA), also known as the No Child Left Behind (NCLB) Act, CSRD 
is now known simply as Comprehensive School Reform or CSR (as it is no longer a 
“Demonstration” program). The program is regulated by the new CSR authority in Part F of 
ESEA’S Title I and the Fund for the Improvement of Education. 
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Need for the Study 

SEDL has conducted research on the efforts of several CSR programs in the states that SEDL 
serves: Arkansas, Louisiana, New Mexico, Oklahoma, and Texas. Through several articles 
(CSRD Connections, SEDL Letter) and publications, and through the creation of the CSR Web 
site (http://www.sedl.org/csrd/), SEDL has been involved in research and evaluation, training, 
and information dissemination on CSR programs. 

In 2003, SEDL contracted with Academic Information Management, Inc. (AIM), to conduct a 
one-year status/evaluation study of CSR programs after the three-year federal funding cycle 
ended. The primary purposes of this study were to ascertain the status of education reform 
models formerly supported by CSR funds and to assess the long-term status of CSR programs 
implemented in the five states that SEDL serves. This report addresses two main research issues: 

1) To assess the overall implementation status of CSR programs after federal funding ended 
and the specific reasons that schools chose to continue or discontinue their programs. 

2) To assess the actual and perceived impact of CSR programs on student performance, staff 
capacity, and other school-level outcomes. 

The goal of any CSR program is to produce long-term, lasting reforms in schools that result in 
sustained improvements in student performance. Assessing program continuation after start-up 
funding has ended is essential to understanding the factors that affect program continuation, 
including funding sources, program support, staff training, and model provider support. 
Researching, identifying and understanding the successes achieved by these programs is 
important in encouraging schools to continue investing in these long-term schoolwide reform 
efforts. 

CSR Models and the Schools Involved in this Study 

This report does not provide in-depth descriptions of the models implemented in the schools 
surveyed for this study. Evaluations of the specific reform models are beyond the study’s scope. 
Many organizations have cataloged and described these models. The SEDL Web site provides 
links to several of these organizations. A publication by the American Institutes for Research, An 
Educator’s Guide to School Reform (1999), catalogs 24 schoolwide reform models and the 
National Clearinghouse for School Reform and the Northwest Regional Educational Laboratory 
produced The Catalog of School Reform Models (last updated in September 2002). This 
publication provides descriptions for 64 reform models. 

Originally, 278 schools implemented a variety of CSR models in the states that SEDL serves. Of 
these, 268 implemented a CSR model for at least the three-year funding cycle. Approximately 5 1 
CSR models were used. The top five models implemented were Accelerated Schools, Help One 
Student to Succeed (HOSTS), Success for All, Advancement Via Individual Determination 
(AVID), and Direct Instruction. Appendix A lists all the reform models implemented in these 
schools. Table 1 displays, by state, the number of schools surveyed and the number of models 
identified in the returned surveys. 
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Table 1. Number of Schools in Study and CSR 


Models Identified 


State 


Number of Schools Included 
in Study 


Number of Models Identified in 
Survey 


Arkansas 


24 


12 


Louisiana 


54 


20 


New Mexico 


21 


8 


Oklahoma 


32 


6 


Texas 


137 


37 


Total 


268 


51 



The study included 163 elementary schools, 57 middle schools, 31 high schools, and 17 schools 
with multiple grade levels (e.g., K-8). The number of schools in the study differs from the 
original number of schools participating in CSR because some schools did not complete the 
three-year funding cycle. 

Study Design 

To assess the status of reform models formerly supported by CSR program funds, this study 
attempted to determine the extent to which models were fully implemented and the elements that 
led to program continuation. At the time of this study, the original funding from CSR had ceased. 
This circumstance has complicated the ability to obtain data because, in some cases, the 
personnel originally associated with the CSR program are no longer with the school that 
implemented the model. In addition, programs often did not retain data or other information that 
would be pertinent to this study. 

To obtain the most useful information possible, given limited data availability in many cases, 
AIM used a three-pronged approach. TTie study involved (1) mail surveys to 268 participating 
schools in the five states that SEDL serves; (2) follow-up phone calls to a small sample of 
schools to clarify information provided in surveys; (3) site visits to a sample of those schools; 
and (4) published data. The three viewpoints lent rigor to the study’s information sources and 
contributed uniquely to its findings. General information regarding the study methodology, such 
as the number of surveys sent and returned, the number and location of on-site visits, and sources 
for published data are included in this section. Analysis and synthesis of the collected 
information appear later in this report. 

Limitations and Generalizability 

There are three primary limitations to the generalizability and applicability of the information 
provided in this report. The first is the survey return rate. Although a survey was distributed to 
all programs that were funded during 1998-2001, not all responded. There was an overall return 
rate of approximately 40 percent; however, the return rate varied by state. While the overall 
return rate is sufficient to produce reliable findings, there is a potential bias in that programs that 
did not return a survey may have a unique viewpoint related to discontinued programs. 

Mitigating this bias, however, is that several surveys were returned from schools that had 
discontinued the CSR program. 

A second limitation concerns the lack of student performance data directly attributable to schools 
that implemented a CSR program. The availability of student performance data in Texas schools. 
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however, where the majority of CSR programs were implemented, provides sufficient data to 
examine the general impact of the programs. 

A third limitation regards the amount of time that had passed before information was collected 
concerning the programs. In some cases, two years had passed since CSR funding ended. In 
these cases, school administrators had to recollect events that had occurred two years before. 
While many schools still employed administrators that were associated with the original CSR 
programs, some schools reported that the school or district no longer employed persons familiar 
with the program. 

Surveys 

To assess the current status of each CSR program, researchers developed two types of survey 
questiormaires. The first, “Survey Yes,” was designed for schools still implementing the CSR- 
^ded model at least partially as of 2002-03. “Survey No” was developed for schools no longer 
implementing the original CSR-funded model. (Questiormaires for both surveys are included in 
Appendix D.) All schools received both types, with instructions for participants to select the 
appropriate questioimaire. Before mailing, a code was entered on each questioimaire that would 
allow identification of the respondent’s state and the model the school had implemented. Each 
survey package included a separate card for respondents to complete if they were willing to 
provide additional information on their school’s CSR program. Researchers conducted follow-up 
telephone interviews to some schools that were willing to supplement the information received in 
the surveys. Approximately 20 phone calls were made to clarify information received in the 
survey. 

The munber of surveys distributed corresponds to the number of programs and addresses 
provided by each state education department contact person. The number of schools identified in 
2003 differs slightly fi"om the number of programs originally funded, based on contact 
information provided by each state. 

The departments of education for each of the five states provided a cover letter explanation for 
the survey packages, which were mailed to schools in Arkansas, Oklahoma, New Mexico, and 
Texas in March 2003. Sample letters are included in Appendix B. These schools returned their 
surveys in April. Louisiana surveys were distributed in early April and returned by the end of the 
montL Surveys were sent to the school and to a district contact person. Information for the 
program contact person in the state department of education, SEDL, and AIM was included for 
respondents who might need help or clarification. Each package also included a self-addressed, 
stamped envelope for returning the completed questiormaires to AIM, where response 
information was entered into a spreadsheet program for further analysis. Open-ended (or fi-ee 
response) questions were tabulated and incorporated into the analysis. 

Survey Return Rate 

Table 2 shows, by state, the number of surveys that were distributed and returned. In total, 268 
surveys were distributed and 106 were completed and returned, for an overall return rate of 40 
percent. (In the opinion of the evaluation team, a return rate of about 40 percent is what could be 
expected fi-om programs that have not received CSR funds for two years.) The rate of return 
varied fi"om state to state. Texas schools had the lowest return rate (23 percent), followed by 



Louisiana (48 percent). Higher return rates were obtained from Arkansas (75 percent), New 
Mexico (62 percent), and Oklahoma (56 percent). 



Table 2. NumI 


t)er of Surveys Distributed and Returned 


State 


Number of 
Surveys 
Distributed 


Surveys 

Returned 


Overall 

Percentage 

Returned 

(%) 


Percentage Agreeing 
to Be Contacted by 
Researchers (%) 


Arkansas 


24 


18 


75 


75 


Louisiana 


54 


26 


48 


31 


New Mexico 


21 


13 


62 


54 


Oklahoma 


32 


18 


56 


47 


Texas 


137 


31 


23 


83 


Total 


268 


106 


40 


53 



Table 2 shows that more than half of the schools responding to the survey returned a postcard 
indicating their willingness to provide further information about their CSR program. There is 
variation by state in the willingness to provide further information to researchers about CSR 
programs. (While not strictly part of the survey results, this level of willingness indicates that the 
respondents were interested in providing useful information about their schools’ programs and 
suggests that many may be program proponents.) r -'i ■ 

Respondents did not answer all survey questions. As a result, the numbers of total responses per 
question differ. Due to rounding, percentages will not equal 100 in all cases. 

Survey Respondents 

Table 3 shows that the majority of respondents, nearly half, are principals; whereas, nearly one- 
quarter are program coordinators. Nearly two-thirds of respondents have been employed at the 
school for more than five years, and fewer than 10 percent had been at the school for less than 
two years. The majority of respondents were likely to have been at the school during the time 
period when the CSR program was in place and to have an understanding of the models that 
were implemented. Tables 2 and 3 combine information from both Yes and No surveys. The 
majority of the remaining tables report findings for the two survey types separately. An analysis 
was not conducted to determine differences in opinions between principals and other school 
staff. Almost 60 percent of respondents were principals or assistant principals. 
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Table 3. Classification of Respondents b] 


Number of Years in the Scl 


lool 


Position 


Less than 
2 Years 


3 to 5 
Years 


More than 
5 Years 


Row 

Total 


Col 

Percent (%) 


Principal 


5 


16 


31 


52 


49.1 


Program 

coordinator 


0 


6 


16 


22 


20.8 


Teacher 


2 


3 


7 


12 


11.3 


Assistant 

principal 


1 


4 


6 


11 


10.4 


Other 


1 


2 


6 


9 


8.5 


Column total 


9 


31 


66 


106 


100.0 



Site Visits 

To supplement the mail surveys of CSR schools, researchers conducted site visits in each 
participating state. Fifteen sites were selected to receive visits from a group of schools that 
indicated a willingness to provide additional information for this study. The number of visits was 
proportional to the number of programs in each state, except in Louisiana, where only one site 
visit was conducted. School sites were selected to reflect different geographic and/or student 
demographics in these states (except Louisiana). CSR state department representatives were- - 
asked to participate in the site visit selection process. To obtain) different view points, site visits 
included schools where the CSR program was still in place and school where the program model 
was discontinued. A variety of CSR models were included in the sites selected. Urban and rural 
school districts were visited. The sites visited are listed in Table 4 
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Table 4. Sites 


deceiving Visits as 1 


Part of CSR Si 


tatus Study 


State 


District 


Grade 

Span 


Model(s) 


Current 

Program 

Status 


Arkansas 


Jonesboro 


Elementary 


Success for All 


Continuing 


Arkansas 


Little Rock 


Elementary 


Reading Recovery, 
Futurekids (changed to 
AlphaSmart) 


Continuing, 
with Early 
Literacy 


Louisiana 


Gretna 


Elementary 


Co-Nect 


Continuing 


New Mexico 


Albuquerque 


Elementary 


Design for Literacy 


Continuing 


New Mexico 


Bernalillo 


Elementary 


Success for All 


Continuing 


Oklahoma 


Lawton 


Elementary 


Success for All 


Continuing 


Oklahoma 


Tulsa 


Middle 


Integrated Teaching 
Instruction, Great 
Expectations (changed to 
Success for All) 


Discontinued 


Texas 


Austin 


High 


Advancement Via 
Individual Determination 
(AVID) 


Continuing 


Texas 


Cypress-Fairbanks 


Middle 


Creating Independence 
through Student-owned 
Strategies (CRISS), 
AVID 


Significant ■ 

portion 

continues 


Texas 


Edinburg 


High 


AVID 


Continuing 


Texas 


Fort Bend 


Middle 


Second Generation 
Effective Schools, 
Mastery in Learning, 
Second Chance Reading, 
Just Read! 


Partially 

continuing 


Texas 


Lubbock 


Junior High 


AVID 


Continuing 


Texas 


Plainview 


High 

(dropout 

recovery/ 

prevention) 


PLATO 


Continuing 


Texas 


San Antonio 


Middle 


Co-Nect, changed to 
Problem-Based Learning 


Discontinued 


Texas 


Ysleta 


Elementary 


El Paso Collaborative for 
Academic Excellence 


Continuing 



The AIM site visit team conducted a one-day site visit to each of the 15 participating schools. 
Each school received a copy of the interview protocol prior to the visit to make the process more 
efficient. During the site visits, several staff members were interviewed: principals, program 
coordinators, teachers, district administrators, and others familiar with the CSR program. The 
four members of the AIM site visit team were all experienced in interviewing school district 
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personnel. Although the interview protocols aided in guiding and focusing their interview 
processes, the team explored additional topics, as appropriate. Appendix C presents summaries 
of each site visit. Appendix D includes copies of the interview protocols. 

Analysis of School Performance Data 

To assess the overall performance of schools that received CSR funding, researchers analyzed 
school performance data by state. All states in the study have published longitudinal student 
performance data. However, not all available data were useful for this study. In three states, 
appropriate grade level data were not available at the time of this study’s publication, or the 
available data were based on tests from different grade levels or where the passing standard had 
changed. Several schools could not provide local objective performance data for the years that 
the program was implemented. 

School performance data from Arkansas were available at the school level. However, changes in 
the grade levels tested over the time period of interest limited the utility of the information. The 
most complete set of data was available in Texas. The 14-year old Academic Excellence 
Indicator System (AEIS) yields a relatively consistent set of data. The AEIS data allowed 
researchers to examine test scores, advanced course completion, and dropout and attendance 
rates. This information, provided by the Texas Education Agency, was taken from complete 
AEIS reports for all campuses in the state. These data are discussed in the performance chapter 
of this report. ■’i*- 
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2 . 

Program Implementation Status 



This chapter presents findings from the three study components — surveys, site visits, and 
analyses of published student performance data. As noted previously, the study sought to 
ascertain the current program implementation status of education reform models formerly funded 
by CSR. Implementation status is presented first, followed by a summary of reasons for 
continuing programs reported by schools with models still in place. Finally, a summary of 
reasons for model discontinuation is presented. 

Table 5 shows that 79 schools returned surveys indicating that models formerly funded by CSR 
are still being implemented (Survey Yes), whereas 27 schools returned surveys indicating that 
models are no longer used (Survey No). Survey No results should be interpreted with caution 
because only 27 such surveys were returned. On the other hand, with three times as many 
respondents, findings from Survey Yes have a greater assurance of applicability. 



Table 5. Program Implementati 


ion Status 






State 


Number of 
Surveys 
Distributed 


Surveys 

Returned 


Programs Still in 
Place (Number of 
Survey Yes 
Returned) 


Programs No Longer 
in Place 

(Number of Survey 
No Returned) 


Arkansas 


24 


18 


18 


0 


Louisiana 


54 


26 


14 


12 


New Mexico 


21 


13 


11 


2 


Oklahoma 


32 


18 


13 


5 


Texas 


137 


31 


23 


8 


Total 


268 


106 


79 


27 



Respondents rated the extent to which models previously funded by the CSR program, as 
originally implemented, were still in place in each school in 2002-03. Table 6 shows that more 
than two-thirds of the schools that are still implementing models are continuing to use the 
models as originally implemented. Another quarter of respondents reported that about half of the 
model is still in place. Although the majority of respondents continue to implement the model 
two years after CSR funding ended, some have made significant modifications and have 
continued to implement only certain aspects of the original model. 
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Table 6. Percentage of CSR-funded Models Still Being Implemented in Schools 



Status 


Number 


Percent 


Entire CSR program is currently in place 


56 


69.1 


About half of the CSR program is currently in place 


19 


23.5 


Only a small portion the CSR program is currently in place 


6 


7.4 



In schools where CSR-fiinded models are still in place, 71 percent (55 schools) reported that the 
staff person responsible for originally implementing the CSR program is still employed by the 
school, whereas 22 schools indicated that this staff person was no longer there. Two respondents 
reported that they did not know whether the staff person who originally implemented the 
program was still employed by the school. By contrast, half of respondents from schools where 
CSR programs have been discontinued said that original staff members responsible for 
implementing the program were still employed in that school, and half said that they were no 
longer there. Although this would seem to suggest that CSR programs are less likely to continue 
if the people who originally implemented them are no longer with schools, it cannot be 
absolutely stated that this is the case, as there are many factors (such as funding) that may also 
strongly determine whether models continue to be used. 

Percentages of respondents indicating that their school’s CSR-funded model is istill in place in its 
entirety vary among states. Table 7 shows that slightly more than half of Oklahoma respondents 
said that models were still in place, whereas 75 percent of respondents from Arkansas said that 
they were still in place. 



Table 7. Program Status by State for Programs Still in Place (Survey Yes) ^ • 



State 


T)riginal Progi^miin § 


ii AboutiHalf injPlace 

# ■ -jS. 


, Small Port 


ionjin Place 


Number 


Percentage 

(%) 


Number 


Percentage 

(%) 


Number 


Percentage 

(%) 


Arkansas 


12 


75.0 


2 


12.5 


2 


12.5 


Louisiana 


9 


69.2 


4 


30.8 


0 


0 


New 

Mexico 


7 


63.6 


4 


36.4 


0 


0.0 


Oklahoma 


6 


54.5 


4 


36.4 


1 


9.1 


Texas 


22 


73.3 


5 


16.7 


3 


10.0 



Reasons for Program Continuation 

Through the surveys and the site visits, researchers documented several reasons for model 
continuation. Table 8 lists the reasons provided for program continuation by Survey Yes 
respondents and people interviewed during site visits as to why their school’s model is still in 
place, either in whole or in part. (Survey respondents were given the opportunity to check more 
than one reason they believe the reform model is still being implemented.) 
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The most important reason for continuation reported by survey respondents relates to successful 
student outcomes. The second most important reason is the CSR program’s alignment with 
school goals. 



Table 8. Reasons Given for Program Continuation (Survey Yes and Site Visits) 



' Survey Yes' ■' .1 






Reasons 


N Times 
Checked 


N of “Most 
Important” 


Successful student outcomes 


72 


45 


CSR program aligned with school goals 


61 


8 


Funding provided by local (other) funds 


59 


2 


Strong teacher and administrator support 


52 


6 


Strong parental support 


48 


0 


: Site;Visits^rp|:j|^pf|| 






More cohesive and comprehensive curriculum 


Increased teacher accountability 


Large investment in time and money 


Improved staff development 


Desire to give program time to show results after significant three-year investment. 


Opportunities for students that otherwise would not be available 



Responses to open-ended survey questions support the findings in Table 8. All respondents who 
answered the open-ended questions reported that successful student performance measures 
(improved student test scores, increased class participation) were the main reason that models 
continue to be implemented. According to one respondent, 

“. . . Improved student performance and alignment with school goals are both most important . . 
.” Another indicated that the model helps the school meet its accountability growth targets. 
Several respondents said that improved student performance was one main reason that funding 
was still available. Regarding strong staff and teacher support, one respondent noted, 
“Collaborative teaching strategies have been taken on by many teachers.” 

Site visits confirmed the findings reported above. Several interviewees said that improved 
student success is the primary reason for a model’s continuation. A number of principals, 
teachers, and program facilitators strongly believe that the use of these models has improved 
overall student achievement. Interviewees also noted that using a particular model has improved 
student discipline and increased parental involvement. One teacher said, “It’s a lot of hard work, 
but we’re seeing results. It’s sequenced in a way that we know what we’re doing and we know 
what we’re looking for.” The site visit summaries included in Appendix C provide comments 
that are more detailed. Detailed, expanded site visit summaries were also prepared as a separate 
document. 

Other important reasons for continuation emerged from the site visits. The comprehensive nature 
of many reform models requires that a school implementing it must make a large investment. 
Programs such as Success for All (SFA) and Design for Literacy require whole school reform. 



Some schools visited by AIM had been completely restructured. Changes included curriculum 
restructuring, staff development and teacher allocations, scheduling, and establishing program 
libraries. In several cases, the CSR-funded model had been used to develop the school’s 
performance goals. 

After large investments of both time and money, respondents in many schools said they lacked 
the desire or the energy to start a new program. For some models, large investments already in 
place — instructional materials, classroom arrangements, staff development, and staff and 
parental support, for example — ensure continuation. Some interviewees said that there were “no 
plans whatsoever” to change or to discontinue implementing their school’s CSR model. One 
principal in New Mexico stated that, with or without funding, “we are committed to continuing 
this program.” In schools with such commitment, staff members believe that CSR models have 
positively impacted the instructional culture of the school. A coordinator from an Oklahoma 
school indicated that the SFA program has become “internalized” there. Over time, the teachers 
have become fully aligned with SFA. When school persoimel hire new teachers, they interview 
for SFA compatibility. The SFA model providers have become their friends. According to this 
coordinator, “no matter what,” the school will continue to use the model: “We are SFA.” 

Despite a significant investment and a strong commitment to continuing a model, several site 
visit interviewees reported that their schools have not yet achieved the level of improved student 
performance that they had expected. In many instances, student performance on state-level 
exams does not necessarily reflect a model’s success. Nevertheless, the majority of interviewees 
felt that use of the model selected for their schools would eventually increase student 
performance. In these instances, school staff members noted that it has taken two to four years 
for these comprehensive models to be fully implemented. In some schools, interviewees wanted 
to give the program a “five-year test,” allowing the original group (e.g., first grade students) to 
go all the way (e.g., through grade 5), being taught via the model. For one school implementing 
the Advancement via Student Determination (AVID) program, a college-bound program for 
entering high school freshmen, 2003 marked the first AVID graduating class as well as the year 
that CSR funding ended. Although CSR programs were ending after their allotted time spans, the 
majority of the site visit schools had established long-term commitments to the model (and to the 
students participating in the model) that went beyond the three-year funding period. This 
commitment to continuation was evident, regardless of whether additional funding had been 
secured for continuation of a school’s model. 

Site visit interviews also revealed that continuation is related to administrative and school staff 
support. In many instances, school principals were fully committed to ensuring continuation of 
the CSR model and, in all instances where models were still in place, school staff members felt 
that they had district support to continue them. 

Some principals and teachers noted that efforts toward school restructuring have resulted in 
better schools. Principals said that teachers had become more accountable. For some models, 
even though the implementation transition period was difficult for them, teachers perceived 
themselves as being better teachers and felt that the overall school curriculum and instructional 
approach were more comprehensive and cohesive. Several teachers said that they felt less 
isolated because more coordination now exists as a result of implementing their selected model. 
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Some interviewees had a strong commitment to CSR models not involving whole school reform. 
For example, AVID and PLATO programs — ^not currently targeting an entire school 
population — are perceived by staff members as positively affecting performance of students who 
participate. Staff members observed that these programs provide opportunities to students that 
would not otherwise be available. The majority of interviewees felt that they would like to see 
implementation of the model expand to include the entire school. 

Respondents from some schools, although keenly aware of the lack of funds, felt that these 
programs were so ingrained in the school structure, system, and culture that it would be 
impossible to “revert back” to a program that did not include the reform model. In these 
instances, district funds, along with other local and federal funding sources, continue to support 
the models. 

Reasons for Program Discontinuing 

As noted, respondents from schools no longer implementing their CSR model were asked to 
return a version of the survey questionnaire tailored for schools that discontinued the CSR model 
(Survey No). Of the 106 surveys that were returned, 27 were the Survey No version. These 
respondents provided the most important reasons that their schools had chosen to discontinue the 
models implemented through the CSR program. Only two of the schools AIM visited had 
discontinued their models (Co-Nect, Great Expectations, and Integrated Teaching Instruction). 

Table 9 provides a summary of respondents’ reasons why models were discontinued. The two 
options selected most frequently relate to program funding and costs; “Did not obtain local (or 
other) funding” and “too expensive to continue.” Following in importance, respondents selected 
“Did not have support of teachers or administrators,” and “Chose another program.” 

A few respondents indicated that, over time, the model originally selected for their school proved 
to be inappropriate for the school. One noted that the program was discontinued because the 
materials were never made available in Spanish, stating, “I do believe that if a program had been 
chosen that included a Spanish component, the CSR grant would have had positive effects in the 
areas of student achievement and parental involvement.” 

The number 1 reason (improved student performance) Survey Yes respondents gave for 
continuation of their model (Table 8) appears toward the bottom of the list of reasons given by 
Survey No respondents when asked why their school’s model was no longer in place (Table 9). 
Therefore, some discontinued models may actually have been producing desired student 
outcomes (see the student performance section of this report). 
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Table 9. Reasons Given for Program Discontinuation (Survey No and Site Visits) 



Reasons 


N Times Checked 


N of “Most 
Important” 


Did not obtain local (or other) funding 


14 


4 


Too expensive to continue 


8 


3 


Did not have support by teachers of administrators 


4 


0 


Chose another model 


4 


0 


Did not obtain good support from model provider 


3 


1 


Did not have strong parental support 


3 


0 


Did not achieve successful student outcomes 


2 


1 


Do not know why program was discontinued 


1 


0 


Do not need any longer 


1 


0 


Did not have adequate leadership 


1 


0 


Did not align with school goals 


0 


0 



Responses to open-ended survey questions also indicate that lack of funding and program 
support are the two most important reasons for discontinuing a model. Reasons such as change in 
district leadership or school board members, or the hiring of a new school principal not 
supportive of the model, were given to explain the model’s discontinuation. Several respondents 
reported that their school’s model was too expensive to support. One noted, “For the money 
spent, we did not get the results we expected.” Another reported that the school “did not obtain 
adequate or quality support from the model provider.” 

Two site visits were conducted at schools with discontinued CSR model. A Texas school 
implemented Co-Nect during the first year of its grant. Thereafter, the school used Problem- 
Based Learning (PBL), a reform model supported by the Region 20 Education Service Center. 
Federal CSR fimds and Title I funds were used during the first year to purchase computers, 
software, materials for classroom libraries, and professional development. Second- and 
third-year funds were used for computers and software and to implement PBL. 

The site visit team observed several differences in this school compared to other schools with 
models still in place. The two most important differences reflect the findings from the Survey 
Yes results shown in Table 8. The reform model in this school was not comprehensive, and it did 
not receive strong support from the school’s administration. Although several teachers were 
trained to implement PBL, only a few continue to use these strategies in the classroom. 
Implementation of this model did not require the school to undergo a comprehensive 
restructuring, nor did it require the school to significantly alter its instructional approach. The 
same level of commitment, involvement, and school transformation observed with some 
programs like AVID, Success for All, and Design for Literacy was not evident in this school. 
When new campus and district administrators began revising goals, the school was not able to 
maintain the model. According to the coordinator, it “fell by the wayside” because the district 
leadership did not embrace the school’s initiative. This school represents a contrast to other 
schools discussed previously, where there was a strong commitment to the CSR program, and 
where the school staff worked as a team to implement CSR model. This school did not 
experience a strong, early and ongoing commitment to the CSR program. 
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The second site visit was conducted at a Oklahoma school that discontinued the model primarily 
for reasons related to funding and the district’s belief that students had not made sufficient 
progress. In addition, the district implemented a local reform model, the Tulsa Reads program, to 
try to put all schools on “the same page.” 

Respondents from schools that had discontinued their models were asked to provide information 
about when their models were most recently used in the schools. Table 10 presents the years 
reported. 



Table 10. Year CSR Model Was Last Used 



School Year 


Number of Respondents 


1999-2000 


9 


2000-2001 


11 


2001-2002 


2 


2002-2003 


1 



Funding for Continuation 

Funding has affected models implemented through CSR grants in various ways. During site 
visits, all interviewees agreed that, without their three-year CSRD program grants, it is unlikely 
that they would have ever implemented models. The initial three-year grant allowed schools to 
make much needed, schoolwide investments involving materials, restructuring, and staff 
development. On average, grantees received approximately $50,000 per year. This level of 
funding allowed schools to implement expensive reform models. 

When Survey Yes respondents were asked how post-CSR continuation of their schools’ models 
was funded, they responded somewhat incompletely. For example, when describing their 
schools’ funding sources, respondents often divided percentages among the three categories 
(local, federal, and other) in a fashion that did not add to 100 percent. On average, their reported 
funding sources were split thusly: 48 percent local, 38 percent federal (i.e.. Title I), and 10 
percent other funds (e.g., grants). 

During site visits, opinions among interviewees varied regarding the degree to which diminished 
funding has eroded the effectiveness of CSR models. Although most agreed that the models are 
expensive to maintain, they also indicated that the expenses associated with the models have 
decreased since the first years of implementation. That is, current costs are less than initial 
implementation costs. A few respondents indicated that, as their school’s model continues from 
year to year, it costs less to maintain. According to one, “initial costs have dropped dramatically 
...” Another noted that “next year, the program will be placed on maintenance; this is less 
expensive, since the use of a consultant will not be done on a monthly basis.” 

Creative hiring constitutes one of the ways schools keep models in place. One school hires four 
quarter-time retired certified teachers to ensure that students receive the one-on-one reading 
instruction required by programs such as Design for Literacy. A district secured private funding 
(a grant from the Dell Computer Company) to continue and expand the AVID model to all 
district high schools. 
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On the other hand, although many principals felt that their school’s model had not yet suffered 
from the lack of funds, they are questioning how they will be able to sustain the model over time. 
All have begun to “cut comers” to ensure that the model can continue. Slightly increasing 
student-teacher ratios, decreasing staff development, and purchasing fewer instmctional 
materials are the top three areas that principals and coordinators reported modifying. Some staff 
members fear that models will be changed to such an extent that their effectiveness will 
diminish. 

Respondents from schools still implementing their CSR models were asked how the models have 
changed since CSR funding ended. The majority reported that, except for funding sources, the 
intent and purpose of their models have not changed. Most, however, noted several changes, the 
majority related to staff development, materials and resources, and staff allocations. While most 
have supplemental funds (combining school, district, and federal Title I funds) to continue 
implementation, schools typically do not have the same level of financial support as when their 
models were fully funded. As one respondent pointed out, “Without funding, we can’t buy new 
materials, engage in support, expand or replace the program, or have professional development 
for the program.” 

Several respondents noted that staff development was sharply curtailed or no longer available. 
They reported that new faculty members are not as well trained and not as familiar with the 
models’ details as the teachers who were originally trained. A few of their responses on staff 
development appear below: 

© The assigned external support personnel are no longer available. 

© There is limited funding available for training and staff development. 

© Hardly anyone can attend the annual conference for professional development. 

© As teachers leave our school, it is difficult to provide the same training to [their] 
replacements. 

© Funding is not available for training that was previously [provided] by the 
program. 



With regard to teacher and staff allocations, schools have made several adjustments to 
accommodate their models after CSR funding ended. One respondent reported that fewer tutors 
are available; another reported that the program facilitator position has been reduced to half time. 
Another said that the district has hired an in-house program coordinator. Other respondents 
commented on these adjustments: 

© The team coaching is one area that is missing. 

Implementation has changed because there is less support. 

© District pays the salary of one teacher; our school uses Title I funds to pay 
for teacher assistant. 

© Teachers have changed the amount of time they spend in using the CSRD 
program. 



BEST COPY AVAILABLE 



16 



Respondents’ comments also show that, overall, fewer resources and materials are available to 
continue implementing CSR models: 

© Additional materials and resources have not been purchased due to the lack 
of funds. 

© Books are not being added to the library; we only replace books as they 
wear out. 

© We can no longer provide the enrichment field trips. 

© We can no longer buy new equipment or repair what we have. 

Some respondents indicated that the “mood” or “attitude” has changed because participants 
know CSR models are at risk of discontinuing because funding is low or absent. One respondent 
noted, “The program will have to be discontinued primarily due to major cuts in state funding.” 
Another said, “We are desperately seeking extra funds to keep the program going. . .” Some 
expressed concern that the current level of funding does not allow for adequate implementation. 
According to one, the district has “picked up the funding, which is less than originally funded. 
This has restricted the effectiveness of the program. . .” A few respondents also observed that 
teachers have become less motivated to be part of the program. 

Schools that chose not to continue implementing the CSR model were asked what would have 
prevented model discontinuation. The consensus was that additional' funding would have allowed 
some of the models to continue. Although funding was the primary issue, respondents also 
identified others. A few respondents cited the lack of effective or consistent implementation 
and/or lack of support fi"om the model provider. 

Although funding was the most often provided reason, other factors also contributed to program 
discontinuation. Many schools that continued the program incurred significant expenses 
(although reduced fi"om the first year of implementation) after funding ended. Site visit findings 
and survey comments indicate a belief that if the CSR program was strong and successful, so 
was the motivation to continue to fund the program after CSR funding ended. School that 
continued the program indicated several funding sources were used to continue the program 
including Title I funds, grants and local funds. In general, school staff suggested that if the 
program was strong, funding would be available. There were, however, some schools that 
implemented models that are expensive and that do not necessarily have a reduced cost over 
time. This expense is related to the lack of transfer of knowledge fi-om the model provided to the 
local school. Over time, it is presumed that a CSR program will build local school capacity that 
would result in a reduction of direct support fi"om the model provider. 

Implementation Level-Institutionalization 

Both surveys examined the extent to which the three stages of school improvement had been 
attained. As shown in Table 11, 100 percent of respondents to Survey Yes reported attaining 
“Initiation” and “Implementation,” and 76 percent reported reaching the “Institutionalization” 
phase. This 76 percent is a close match to Ihe 70 percent of models that remain in place. In other 
words, once the model becomes “institutionalized,” it will be continued, assuming that funding is 
available. An examination of shaded cells on the right side of Table 1 1 reveals an apparent 
“diagonal,” indicating that it takes time to obtain the final “Institutionalization” stage. Only 2 
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percent of schools reached that stage during their first year. In fact, nearly 40 percent of schools 
required four years to attain institutionalization for their CSR model. (The number of 
respondents to this question varied between 55 and 67 out of a total of 79 Survey Yes 
respondents.) 



Table 11. School Improvement; Level Attained (Survey Yes) 



Stages of School 
Improvement 


r- ''Stale- ■;iff 

j completed? (Vaji 


lii what year was stage completed? 9%) J 


Yes 


No 


1999 


2000 


2001 


2002 


2003 


Initiation/Mobilization 


100 




■ 45f'-'' 


•37 


15 


3 


0 


Implementation 


100 




16 


33 


/■::iE28.-. V 


18 


5 


Institutionalization 


76 


24 


2 


15 


22 




24 



Table 12 presents levels of attainment for schools that are no longer implementing the CSR 
model (Survey No). Twenty of the 25 respondents to Survey No answered this question. In some 
cases, these respondents indicated that their schools reached the “Implementation” and 
“Institutionalization” phases sooner than schools whose CSR models are still in place. 



Table 12, School Improvement: Level Attained (Survey No) 



Stages of School 
Improvement 


Stage 

^completed? (%|| 


sfln yvhatl^ear wa^stage complete^{%) 


Yes 


No 


1999 


2000 


2001 


2002 


2003 


Initiation/Mobilization 


100 


0 


43t: : 


36 


7 


14 


0 


Implementation 


95 


5 


7 




7 


21 


0 


Institutionalization 


47 


53 


0 


0 


'112 


29 


0 



Chapter Highlights 

o 79 schools returned surveys indicating that models formerly funded by CSR are still 
being implemented 

o 27 schools returned surveys indicating that models are no longer used (Survey No). 
Survey No results should be interpreted with caution because only 27 such surveys were 
returned. 

o The majority of schools that are still implementing models are continuing to use the 
models as originally implemented. A quarter of schools reported that about half of the 
model is still in place. Some have made significant modifications and have continued to 
implement only certain aspects of the original model. 

o The most important reason for continuation reported by survey respondents relates to 
successful student outcomes. The second most important reason is the CSR program’s 
alignment with school goals. 
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o The comprehensive nature of many reform models requires that a school implementing it 
must make a large investment. Some schools visited were completely restructured. 
Changes included curriculum restructuring, staff development and teacher allocations, 
scheduling, and establishing program libraries. In several cases, the CSR-funded model 
had been used to develop the school’s performance goals. 

o Several schools have not yet achieved the level of improved student performance that 
they had expected. Study findings suggest that student performance on state-level exams 
does not necessarily reflect a model’s success. The majority of interviewees felt that use 
of the model selected for their schools would eventually increase student performance. 
Interviewees wanted to give the program a “five-year test,” allowing the original group 
(e.g., first grade students) to go all the way (e.g., through grade 5), being taught via the 
model. 

o Program continuation is directly related to administrative and school staff support. 

o For some schools, efforts toward school restructuring have resulted in better schools. 
Reform efforts produced more accountability, improved teaching strategies, improved 
teacher coordination and communication, and an overall improved and cohesive school 
curriculum and instructional approach. 

o Some schools reported that these programs were so ingrained in the school structure, 
system, and culture that it would be impossible to “revert back” to a program that did not 
include the reform model. In these instances, district fimds, along with other local and 
federal fimding sources, continue to support the models. 

o Site visit findings and survey comments indicate a belief that if the CSR program was 
strong and successful, so was the motivation to continue to fund the program after CSR 
funding ended. 

o On average, funding sources for schools continuing the program include local funds (48 
percent), federal funds (i.e.. Title I, 38 percent), and other funds (i.e., grants, 10 percent). 

o The top four reasons for program discontinuations are: did not obtain local (or other) 
funding, too expensive to continue, did not have support of teachers or administrators, 
and chose another program. 

o Although school staff agreed that the models are expensive to maintain, they also 
indicated that the expenses associated with the models have decreased since the first 
years of implementation. 

o Findings indicate that school staff are questioning how they will be able to sustain the 
model over time. All have begun to “cut comers” to ensure that the model can continue. 
Slightly increasing student-teacher ratios, sharply decreasing staff development, and 
purchasing fewer instmctional materials are the top three areas that principals and 
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coordinators reported modifying. Some staff members fear that models will be changed 
to such an extent that their effectiveness will diminish. 

o One hundred percent of respondents to Survey Yes reported attaining “Initiation” and 
“Implementation,” and 76 percent reported reaching the “Institutionalization” phase. This 
76 percent is a close match to the 70 percent of models that remain in place. This finding 
suggests that once the model becomes “institutionalized,” it will be continued, assuming 
that funding is available. 
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3 . 

CSR Model Components 



In addition to assessing the overall implementation status of models funded by CSR grants, this 
study also surveyed participants regarding the components that form the foundation of their CSR 
programs. Originally, grantees were encouraged to base model implementation on nine CSR 
components. Since No Child Left Behind (NCLB), the new CSR program added two more 
components. Thus, schools that apply for CSR program funding must now address and align 
their reform programs with 1 1 components. A CSR program; 

o employs proven methods and strategies based on scientific research, 
o integrates a comprehensive design with aligned components, 
o provides ongoing, high quality professional development for 
teachers and staff. 

o includes measurable goals and benchmarks for student achievement, 
o is supported within the school by teachers, administrators, and staff, 
o provides support for teachers, administrators, and staff 

o allows for meaningful parent and community involvement in planning, implementing, 
and evaluating school improvement activities, 
o uses high quality technical support and assistance from an external partner with 
experience and expertise in schoolwide reform and improvement, 
o plans for the annual evaluation of strategies for school reform implementation and for 
student results achieved. 

o identifies resources to support and sustain the school’s comprehensive reform effort, 
o has been found to significantly improve, or demonstrates strong evidence that it will 
improve, the academic achievement of students. 

Survey respondents and site visit interviewees were asked how their schools’ CSR model 
implementations incorporated the initial nine components included in the original CSR program. 
Table 13 shows the results from the survey of schools with models that continued after program 
funding ended. 
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Table 13. CSR Components of Models Being Implemented (Survey Yes) 
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No 
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research-based 
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11 


0 


87 


13 


95 
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Use a 
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design with aligned 
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84 
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28 


89 


11 


2 
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83 


16 


1 


87 


13 


91 


9 


2 


Have support 
within the school 


78 


22 


0 


83 


17 


91 


9 


0 


Have parental and 

community 
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55 


40 


5 


71 


29 


76 


24 


5 


Use external 
technical support 
and assistance 


68 


28 


5 


68 


32 


82 


18 


2 


Have evaluation 
strategies 


81 


19 


0 


83 


17 


93 


7 


2 


Coordination of 
resources 


78 


21 


1 


80 


20 


92 


8 


3 



Note: Don’t Know responses are not included in Yes/No category percentages. 



The data in Table 13 show that a large percentage of schools that fully implemented the CSR 
components as part of their models continued to implement the components after funding ended. 
An exception is the professional development component, which, not unexpectedly, was less 
strongly implemented after the three-year funding period ended. Staff members interviewed 
during site visits indicated that as models were implemented fi'om one year to the next, fewer 
teachers needed training. New teachers still needed to be fully trained, whereas continuing 
teachers only needed to be kept informed of new methods and model updates. Still, given the 
turnover rate and percentage of teachers leaving the profession each year, continued staff 
development is important to a model’s effectiveness. (In Texas, for example, the annual turnover 
rate for teachers is about 
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1 6 percent, according to the 2002 Academic Excellence Indicator System; and, according to the 
Texas State Board for Educator Certification, almost half of all new teachers leave the profession 
before the end of their fifth year.) 

Findings fi'om site visit interviews regarding professional development were consistent with 
survey findings. Interviewees said that professional development is one component for which 
activity has decreased since CSR funding ended. Because of diminished funds, most models that 
are still in place use previously trained teachers to train new teachers. Interviewees indicated that 
funds for conferences and workshops are no longer available and noted that it is difficult to stay 
informed and updated. 

When asked whether the CSRD components were effective, a large percentage of survey 
respondents indicated that most components were producing the desired results. Although the 
components “Have parental and community involvement” and “Use external technical support 
and assistance” were rated lower than other components in terms of results, more than 75 percent 
of respondents said that these components had achieved expected results. 

Table 14 presents information regarding CSR program components for schools that chose to 
discontinue implementing their models after program funding ended (Survey No). A comparison 
of data in Tables 13 and 14 indicate that a larger percentage of schools not continuing their 
models also reported not fully implementing several components. While more than 70 percent of 
schools continuing their model reported that they fully implemented the majority of the 
components, approximately 50 percent of schools that discontinued their models also reported 
that they fully implemented most of the model components. In other words. Survey Yes 
respondents were more likely than Survey No respondents to report that all.CSR components . 
were fully implemented. 
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Ta ble 14. CSR Components of Models No Longer in Place (Survey No) 
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42 


50 
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65 


35 


1 



Note: "Don’t Know” responses are not included in Yes/No category percentages. 

The largest difference between responses to the Yes and No surveys was under “Coordination of 
resoirrces.” The next largest differences were found for “Use of effective, research-based 
methods” and for “Use a comprehensive design with aligned components.” All three of these 
components relate to efficiency at least as much as they relate to effectiveness. These data seem 
to reinforce the idea that planning for the implementation of a model and careful selection of the 
approach are critical to its success. 
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Site Visit Findings on Model Components 

During site visit interviews, program coordinators and teachers did not think of their schools’ 
models in terms of the nine CSR components. Although some recalled hearing about the 
components in a workshop, they did not specifically remember implementation of their school’s 
model as having been based on these components. Still, it appears that overall, the CSR model 
components were addressed and implemented. Several examples of how components were 
addressed follow. 

Use Effective, Research-Based Methods 

Although the selection process varied fi'om school to school, most selected a model that had 
previously been identified as research-based. (Models selected by the large majority of schools 
surveyed are included in several publications that characterize these models’ methods as 
research-based.) 

Several schools formed selection committees of teachers and administrators who did their own 
research by visiting other schools — ^typically, those with demographic profiles similar to their 
own students — ^to observe the models being used. These visits helped them identify the models 
most appropriate for their schools. In addition, state departments of education, SEDL, district 
staff members, and model providers offered workshops and presentations for schools selecting 
appropriate models. 

Some schools modified their selected model — despite the model’s basis in effective, research- 
based methods — to better serve their local student population. In such instances, interviewees 
noted that the research was not necessarily conducted at a site where conditions matched the 
needs and circumstances of their school’s students. 

Use a Comprehensive Design with Aligned Components 

The majority of schools visited by the AIM research team had implemented models that used a 
comprehensive design with aligned components. Models such as Success for All, Advancement 
via Student Determination (AVID), and Design for Literacy include specific lessons that align 
with particular instructional and performance goals. CSR models that have remained in place 
after the grant’s end are fully aligned with the school/district goals. In some schools, the CSR 
model goals themselves were adopted as the school’s improvement goals. Discontinued models’ 
goals were not integrated or aligned with overall improvement plans for the school or district. 
Some models — Problem-Based Learning (PBL), for example — do not align well with state 
standardized assessment systems. PBL is better suited to assessment that is portfolio-based. 

Several examples from the site visits demonstrate how CSR models align with state, local, and 
federal goals. In Arkansas, for example, all schools are expected to implement the statewide 
Early Literacy Learning in Arkansas (ELLA) in grade 1 and the Arkansas Effective Literacy 
writing program in grades 2 and 3. In one school that implemented Success for All (SFA), 
teachers can use SFA and ELLA simultaneously because both are guided by the same principles. 
Older students use a state-mandated multicultural reading, thinking, and writing program instead 
of SFA writing. SFA consultants allow teachers to make these adjustments, and they have 
recognized the shift toward the state’s accoimtability system by offering accountability related 
sessions at their conferences. In addition, their focus has changed from strictly implementing the 
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model to achieving measurable student performance. This school has also created documents 
aligning SFA with district benchmarks and with the SAT 9 testing program. 

In another Arkansas school, interviewees noted that the Reading Recovery model they 
implemented with CSR fimds is closely aligned with ELLA. Teachers are pleased with the 
ELLA/Reading Recovery combination. One said, “It’s a lot of hard work, but we’re seeing 
results. It’s sequenced in a way that we know what we’re doing and we know what we’re looking 
for.” 

Oklahoma requires every child read at grade level by third grade. The coordinators at one school 
said that SFA has contributed significantly toward reaching this goal because the model requires 
that assessments be conducted every eight weeks. The SFA assessments, along with the 
Scholastic Reading Inventory (SRI), allow teachers to monitor progress and to make adjustments 
to achieve the state goal for all third grade students. 

In Texas, several models align well with both state and federal goals. AVID and Creating 
Independence through Student-owned Strategies (CRISS), for instance, are consistent with 
district and state goals in that they are designed to lower student dropout rates, increase the 
umber of students taking AP classes and SAT/ACT college entrance exams, increase the number 
of students attending college, and increase the number of students graduating under the state’s 
recommended or distinguished plan. 

The study also examined CSR models that did not align with local or state performance goals. 
After implementing Co-Nect and PEL, one Texas school chose to discontinue both models partly 
because staff members felt that their use did not support state and/or local goals. The program 
coordinator noted that the scope and sequence of the Texas Essential Knowledge and Skills 
(TEKS) curriculum is, and that teachers did not feel comfortable using student-centered PEL 
strategies in light of TEKS. 

Another reason this school discontinued its CSR models was that the incoming superintendent 
had developed new goals for all schools in the district. These goals were viewed as directly 
impacting teaching strategies. Another complicating factor was that the school was in the process 
of being restructured from a grades 6-8 campus to a K-8 campus. Under these conditions, 
teachers did not feel that administrative support would be sufficient for them to continue PEL. 

Provide Ongoing, High Quality Professional Development 

At all schools visited, interviewees reported that staff development — ^particularly during the first 
few years of implementation — ^was critical to the success of CSR models. All schools included 
the staff development component as part of the implementation process for their CSR models. 
During the first three years, while CSR fimds were available, teachers attended to conferences 
and workshops. Several model providers organized in-school training sessions and provided 
contact persons, who visited schools on a regular basis. 

Despite the importance of the staff development component, principals recently noted that this 
training has been significantly impacted by the lack of fimding. Previously, schools were able to 
send teachers and coordinators to annual conferences, but the majority of principals and 
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coordinators said that funds for attending annual conferences presented by model providers were 
no longer available. By the time interviews were conducted, schools had begun to use previously 
trained teachers to train new teachers, and experienced teachers were meeting regularly with 
other teachers for discussions on the model. In one school, for example, staff members met once 
a week to discuss specific issues regarding the model or to hear presentations fi-om colleagues in 
their school or fi'om other school districts. 

Develop Measurable Goals and Benchmarks 

All CSR models involved in this study have measurable goals and benchmarks that are 
incorporated with school and state assessment systems. Several models include periodic student 
assessment with specific benchmarks for improvement. For example, models such as Design for 
Literacy, Success for All, and Reading Recovery include built-in assessments that measure 
student progress. AVID includes specific goals that measure student participation and success. 

A school district’s goals change over time. Often, shifts in goals are driven by demands of 
federal, state, and, in some cases, local mandates. Many interviewees noted that the outcomes 
they originally considered as important have shifted to reflect goals associated with No Child 
Left Behind or their state’s accountability systems. School personnel are learning to develop 
goals and measurable outcomes that simultaneously take into account local, specific school 
needs and state and national accountability requirements. For example, if certain aspects of grade 
4 mathematics have been weak for the past several years, it is appropriate to target needs related 
to grade 4 mathematics instruction, with the understanding that overall mathematics performance 
will still be measured in terms of accountability. Thus, a model with a very narrow focus may 
achieve performance increases that are so limited that the model, although successful, ultimately 
fails the school because it may not be compatible with district, state, or national goals. 

Have Support within the School 

Site visits confirmed that models were best implemented with school and district support. 
Without district support, the use of CSR models was usually discontinued. 

Several principals said that it was difficult to obtain staff support at the beginning of the CSR 
program. Involving staff members in selecting the CSR model helped ensure their support. With 
several models, teachers indicated that they felt intimidated at the beginning of model 
implementation. In Success for All and Design for Learning, for example, teachers must use 
instructional strategies that they may never have used before. Older, more experienced teachers 
felt threatened, and many resisted the change. One principal said, “All teachers were put to the 
test, and some left our school.” With time, however, and after observing the results of improved 
student performance, teachers began to support the use of many of these models. One said, “The 
program sells itself.” During site visits, all teachers interviewed were supportive of their school’s 
reform model and thought of themselves as better teachers because of the CSR program. 

Successful models also require support fi-om teachers and other staff members who are not 
directly involved in the models’ implementations. In one AVID program in Texas, as part of a 
task that teaches organizational skill, students are required to maintain a notebook for each class 
using specific guidelines. Students in the AVID program routinely visit colleges, universities, 
and work sites to learn about different professions. The AVID coordinator in this school said that 
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the model works best when all teachers in the school are aware of its goals and when they 
support the model by providing flexible arrangements for AVID students to make up classes and 
tests. 

Have Parental and Community Involvement 

Responses regarding the parental involvement component varied across site visits. Most 
interviewees noted that although parental support exists for their school’s model, increasing 
parental involvement is still challenging. While parents appear to support CSR models, this 
support does not necessarily translate into increased involvement. Still, some interviewees 
indicated that as a result of CSR program implementation, the school has enjoyed an increase in 
parental support and involvement. One New Mexico school principal said the program had 
increased the number of parents requesting transfers for their children to attend his school. 

Site visits revealed that comprehensive programs do not go uimoticed by parents. Elementary 
schoolwide programs such as Success for All and Design for Literacy involve schedule changes 
and other investments that modify the instructional methods and, in some cases, the school 
culture as it pertains to student learning. These changes appear to be regarded positively by 
parents. 

Interviewees indicated that model providers do not always assist school staff members or provide 
training that involves strategies for increasing parental and community support. 

Use External Technical Support and Assistance 

Interviewees reported that one of the most important components regarding CSR model 
implementation is the use of external technical support and assistance. Similarly, survey 
respondents were generally pleased with the level of support that they received from the model 
providers and said that proper training, particularly in the first two years of implementation, was 
critical to the model’s success. Tables 15 and 16 contain information about the quality of the 
models and their providers. There is a marked difference between responses for the two types of 
surveys. Yes and No. Fewer respondents to Survey No — ^those whose schools are no longer 
implementing the CSR model — said that the quality of the model and support were excellent 
than respondents to Survey Yes. Although lack of funding was listed as a primary reason for not 
continuing the model, other issues related to the quality of the approach and the support provided 
also contributed to model discontinuation. 



Table 15. Model Quality: Program Still in Place (Survey Yes) 



Model and Provider 


Poor 

(%) 
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Good 

(%) 


Excellent 

(%) 


Total N 


Please rate quality of model used for the 
CSR program. 


0 


3 


30 


68 


80 


Please rate quality of support received 
from the model provider. 


1 


1 


33 


65 


80 
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Table 16. Model Quality: Programs No Longer in Place (Survey No) 



Model and Provider 


Poor 

(%) 


Fair 

(%) 


Good 

(%) 


Excellent 

(%) 


Total N 


Please rate quality of model used 
for the CSR program. 


0 


25 


50 


25 


24 


Please rate quality of support 
received from the niodel provider. 


9 


22 


44 


26 


23 



Many interviewees indicated that model support was very good, both in schools that continued 
the program and in schools that did not continue. In some cases, consultants provided informal 
help even after the CSR funding had ended. On the other hand, support from one provider (out of 
all the models implemented) was characterized as less than adequate and sometimes confusing. 

As was true for professional development, some model support, over time, has shifted from the 
model provider to the school. In some successful models, external support (for example, training 
of a master teacher by the model provider) was replaced by internal support (through use of the 
trainer of trainer model, for example), for which teachers whose training was paid for by CSR 
funds now serve as resources for other teachers. Such evidence of institutionalization correlated 
with good outcomes. 

Have Evaluation Strategies 

Strategies for evaluating CSR programs before funding ended were primarily based on 
evaluation requirements stipulated in the CSR grant. Evaluation strategies varied by state and 
school. They typically involved an analysis of state-administered assessments, along with overall 
school staff and parent perceptions regarding the models. Most evaluation strategies were 
informal. Although educators have a broad range of ideas regarding methods and outcomes to 
assess the effectiveness of these models, few have conducted longitudinal, comprehensive 
evaluations of their schools’ models. More detailed information regarding evaluation strategies 
and student performance is presented in the chapter entitled “Measures of Performance.” 

Coordinate Resources 

Coordination of resources is an important component, especially now that schools no longer 
receive CSR funding. All have supplemented their models’ ongoing operating costs with local, 
state, and federal funds. Some schools have also applied for and received additional federal 
grants, and a few have received grants from private organizations. 

Academic Achievement of Students 

To be considered successful, CSR programs must improve student achievement. While some 
schools’ programs have not yet achieved such results, respondents from all of the schools that 
chose to continue their CSR models after funding ended indicated that they believe the models 
will produce improved student achievement. Many school administrators said that they did not 
consider the first three years of implementation to be long enough for changes in student 
performance to become evident. School staff members said that several components must first be 
in place: staff training, staff acceptance of the model, parental support, alignment of the model’s 
goals with school and district goals, and a working relationship with the model provider. In some 
schools, effectively implementing all these components may take one to four years. 
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Some educators believe that improved student performance will become apparent after students 
have been instructed according to the CSR model for a number of consecutive years. In some 
schools, students who are currently in grade 4 or 5 have had such instruction since grade 1 or 2. 
In these instances, after three years of coordinated and cohesive instruction, school staff 
members believe student performance will improve to an extent that it can be measured. In high 
schools, when implementing a model such as AVID, it takes four years to graduate a class of 
students who began participating in AVID as freshmen. In one school, where 2003 was the first 
graduating class for AVID participants, several successful outcomes were highlighted during 
interviews. 

Although performance gains may not be evident until late in a model’s implementation, 
interviewees, in many cases, felt strongly committed to continuing their school’s model, perhaps 
because they believed the model was benefiting students despite the dearth of evidence. Table 8 
indicates that successful student outcomes are the most important reason that some CSR 
programs continued. Despite the importance of successful student performance, some CSR 
programs may be difficult to discontinue after investing so much effort into implementing the 
models, even without immediate evidence of improved performance. It is recommended that a 
carefully considered exit plan be part of adopting any reform model. 

Chapter Highlights 

o The majority of schools that fully implemented the CSR components as part of their 
models continued to implement the components after funding ended. 

o The professional development component was less strongly implemented after the three- 
year funding period ended. As models were implemented from one year to the next, 
fewer teachers needed training. New teachers still needed to be fully trained, whereas 
continuing teachers only needed to be kept informed of new methods and model updates. 

o A larger percentage of schools not continuing their models also reported not fully 
implementing several CSR components. While more than 70 percent of schools 
continuing their model reported that they fully implemented the majority of the 
components, approximately 50 percent of schools that discontinued their models also 
reported that they fully implemented most of the model components. 

o Program coordinators and teachers did not think of their schools’ models in terms of the 
CSR components. Although some recalled hearing about the components in a workshop, 
they did not specifically remember implementation of their school’s model as having 
been based on these components. 

o The majority of schools selected a model that had previously been identified as research- 
based. Some schools modified their selected model — despite the model’s basis in 
effective, research-based methods — ^to better serve their local student population. 

o The majority of schools implemented models that used a comprehensive design with 
aligned components. 
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o Staff development_particularly during the first few years of implementation_was critical 
to the success of CSR models. All schools included the staff development component as 
part of the implementation process for their CSR models. 

o All CSR models involved in this study have measurable goals and benchmarks that are 
incorporated with school and state assessment systems. Several models include periodic 
student assessment with specific benchmarks for improvement. 

o Models were best implemented with school and district staff support. Without district 
support, the use of CSR models was usually discontinued. 

o Although parental support exists for their school’s model, increasing parental 
involvement is still challenging. While parents appear to support CSR models, this 
support does not necessarily translate into increased involvement. 

o The use of external technical support and assistance was reported as an important 

components regarding CSR model implementation is. Survey respondents were generally 
pleased with the level of support that they received from the model providers and said 
that proper training, particularly in the first two years of implementation, was critical to 
the model’s success. 

o While some schools’ programs have not yet achieved such results, respondents from all 
of the schools that chose to continue their CSR models after funding ended indicated that 
they believe the models will produce improved student achievement. > • 
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4 . 

Measures of Performance 



Although the success of a restructuring program or reform model can be measured in a variety of 
ways — enhanced professional development, increased and improved instructional resources, and 
reduction in staff turnover, to name a few — most success measures involve the student. Typical 
student outcomes include increased self-esteem, greater classroom participation, and, of course, 
higher test scores. Several different measure were examined; however, the term “success” is 
dependent on what is trying to be accomplished by each program. This is “internal” success. 
Externally defined success, like graduation rates, may overlap with the internal measures; these 
are often defined by the state. In some cases, the external definition of success may be in conflict 
with the internal goals of the program. 

Programs take a certain amount of time to be effective, regardless of the measurements 
employed. This means attempting to assign an outcome measure (e.g., increased student 
performance on tests) to a particular program is complicated by the length of time the program 
has been in place. In the case of CSR-funded models, an additional factor is that the original 
funding source is no longer in place. The models under review received funding only for three 
years from CSR. Some models are no longer being implemented at all, some are still being used 
in a limited way, and others continue to operate as they did when CSR funding was available. 
Thus when reviewing the outcomes examined in this chapter, the reader should consider the 
degree of implementation and the variety of models used. 

Three data sources were used to examine the performance of CSR-funded models. The first 
source relies on self-reported information gathered from a survey of schools that implemented 
CSR models in the five states included in this study: Arkansas, Louisiana, New Mexico, 
Oklahoma, and Texas. The second source also relies on self-reported data derived from site visits 
to a sample of schools that received CSR grants in the five states. Published, objective data 
constitute the third source. To ensure their accuracy, emphasis is placed on published data. 
Because published data such as statewide criterion-referenced tests are uniform within states, 
longitudinal comparisons among schools are possible when such data are available. However, 
comparisons across states are not possible, given the variety of tests, grade levels, standards, and 
definitions that are used. 

Outcome Measures 

As previously noted, CSR programs are not limited to a particular model; nor is any one goal 
common to all models. Allowing emphasis to be placed where appropriate leads to a diversity of 
approaches and models. While this contributes to the strength of a general program such as CSR, 
it also complicates the issue of measuring student performance. 

A critical precursor for any effort to increase student performance is to have measurable goals 
and objectives so that progress may be ascertained in a concrete, impartial maimer. Measurable 
goals and objectives are key components associated with the CSR program. In schools where the 
model funded by CSR continues to be implemented, 83 percent of survey respondents indicated 
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that they have developed measurable goals and benchmarks. On the other hand, in schools where 
the model has not continued, only 67 percent reported that they had developed such goals. 

Nearly all (98 percent) of respondents in schools where the CSR model is still in place noted a 
positive impact on student achievement. Even in sites where the model is no longer used, 88 
percent reported a positive impact. Moreover, respondents identified a correlation between the 
quality of the model and whether the program was still being implemented in 2003. The quality 
of the model is linked, as discussed in Chapter 3, to the level of student success (although factors 
other than the quality of the model also have an impact). 

Respondents ranked, in order of importance, outcome measures they consider most appropriate 
to gauge the impact of the CSR program. These opinions identify the outcomes that respondents 
believe were addressed by the CSR model they selected to implement in their schools. The extent 
that the outcome list differs, or in non-compatible with state and federal accoimtability standard 
is directly related to the difficulty in continuing the model. Table 17 shows that the second, third, 
and fourth outcomes listed (student engagement, grades, and portfolio) are not commonly 
included in state or federal accountability standards. Tables 17 and 18 contain this information 
for schools with both continuing and discontinued models. The ratings are collapsed into three 
broad categories: number of 1 and 2 rankings (most important); number of 3, 4, and 5 rankings; 
and number of 6 and 7 rankings (least important). 

In Table 17, student test scores were ranked as most important (i.e., ranked first or second) 63 . . 

percent of the time and as least important (i.e., ranked sixth or seventh) 13 percent of the time. 
Although some differences exist between the information in Tables 17 and 18, it is noteworthy 
that the first three outcomes in ranked importance are identical whether or not a model is still in 
use. The last two outcomes are also identical; thus, only student portfolio and attendance were 
ranked differently by respondents from schools with continuing and discontinued models.’ . 



Table 17. Importance of Outcome Measures; Model Still in Place 



Outcome 


Importance 
Rank 1 and 2 
(%) 


Importance 
Rank 3, 4, and 5 
(%) 


Importance 
Rank 6 and 7 
(%) 


Student test scores 


63 


24 


13 


Student engagement 


55 


38 


7 


Student grades 


29 


58 


13 


Student portfolio 


19 


36 


44 


Student attendance 


14 


50 


36 


School completion 


13 


26 


49 


Parental 

involvement 


12 


45 


43 
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Table 18. Importance of Outcome Measures: Model No Longer in Place 



Outcome 


Importance 
Rank 1 and 2 
(%) 


Importance 
Rank 3, 4, and 5 
(%) 


Importance 
Rank 6 and 7 
(%) 


Student test scores 


73 


18 


9 


Student engagement 


47 


37 


16 


Student grades 


40 


50 


10 


Student attendance 


18 


29 


53 


Student portfolio 


16 


37 


47 


School completion 


11 


39 


50 


Parental 

involvement 


11 


47 


42 



As explained elsewhere in this report, the apparent emphasis on parental involvement in 
programs — and, correspondingly, the impact of models on increasing parental 
involvement — seemed to be quite low. On the other hand, the school completion outcome, 
although ranked low in importance overall by respondents in both groups, was of the greatest 
importance in schools serving students who are at very high risk of dropping out, or who have 
been “recovered” from dropping out and hope to complete high school. 

Survey respondents were asked to provide additional outcome measures that could be used to 
measure the success of the CSR programs. Several alternative measures were indicated: 

o School discipline — ^number of student referrals/office referrals; reduction of discipline 
referrals during reading 
o Teacher/staffrcommunity involvement 

o Post-secondary education eiuollment; number of students staying in college 
o Survey results for student and parent attitudes toward reading 
o Number of students taking AP, pre-AP, and advanced classes 
o Number of students obtaining scholarships 
o School environment/climate and classroom management 
o Teacher retention 
o Number of students in honor society 

o Number of students involved in school activities and community service 
o Review of school organization; level of model’s involvement in campus improvement 
plans 

o Increased student input/verbal communication 
o Decreased “teacher talk” time in classrooms 
o Sustainability of model 

Site Visit Findings Regarding Outcomes 

In some cases, obtaining data during site visits was difficult. CSR fimding had ceased two years 
prior to the visits, and many program directors did not keep data specific to the program. Some 
CSR-fimded models had been implemented schoolwide, whereas for other models, only selected 
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students participated. The information reported in this section is based primarily on opinions of 
interviewees and survey respondents (with exceptions noted). 

The mailed survey was structured; it consisted of a set list of outcomes and response options 
(although respondents could list additional outcomes). By contrast, the results from the site visits 
are based on much less directed questions. Although the 1 5 sites visited for the study represent 
about 5 percent of all schools implementing CSR in the five states, the depth of information 
gathered on-site exceeded what was possible from the overall survey. In fact, the responses 
differed from survey responses in interesting ways. Moreover, findings from published data also 
differed somewhat from survey results and from on-site findings. Still, some general patterns 
were evident from those three distinct methods of viewing CSR participants and their 
experiences. 

Table 19 contains the number of times a particular outcome was mentioned by respondents in 
interviews conducted during site visits. For this table, if more than one respondent noted an 
outcome at a particular site, it was counted only once. Note that these data come from a variety 
of schools — ^both continuing and discontinuing implementation of CSR models — and from 
elementary, middle, and high schools. This finding indicates that schools have observed several 
benefits from CSR, many of which are not part of formal state or federal accoimtability systems. 

Table 19. Number and Percentage for Outcomes Rated as Positive and at Least Partially 
Attributable to CSR 



Outcome 


N 


% 


Built teamwork (teachers), decreased turnover, increased reputation of 
school 


11 


73 


Increased student self-esteem, sense of belonging to school, willingness 
to learn 


7 


47 


Helped the student who was far behind 


6 


40 


Created a better student, lifelong learner, self-directed learner 


7 


47 


Improved graduation rates or lowered dropout rates 


5 


33 


Improved achievement test scores 


5 


33 


Helped below grade-level student, but not at rate required by 
state/federal standards 


5 


33 


Worked best for special education or gifted students 


2 


13 


Improved student discipline 


1 


7 


Increased parental involvement 


1 


7 


Improved attendance 


1 


7 



Note. Total number of sites visited = 15 



Student Performance as Determined from Published Data 

For two reasons, this section primarily presents data from Texas schools. First, Texas had the 
greatest number of CSR program sites (more than twice as many as Louisiana, the state with the 
next greatest number). In addition, Texas has compiled many years of comparable data 
pertaining to a variety of outcomes at the school level. Selected school-level data from Arkansas 
are also considered for this survey, but somewhat less information is available for that state than 
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for Texas. Because few schools in New Mexico and Oklahoma were included in this study, no 
formal comparisons are considered, as their data are likely to be quite unstable. It should be 
noted that, in the remaining states, patterns similar to those discussed below for Texas and 
Arkansas have been identified through informal comparisons of districts that implemented CSR 
programs with the overall state. 



Texas 

For at least the past 16 years, Texas has administered some form of statewide criterion- 
referenced test. Data from test scores, along with annual dropout rates, are the primary outcomes 
used to rate campuses and districts under the state’s accountability system. Schools that are rated 
low performing for more than one year are liable for sanctions, including closure of the school. 
The state-mandated test has a strong measurement focus on the state’s instructional elements 
(Texas Essential Knowledge and Skills or TEKS). Data for the comparisons in this chapter are 
based on the version of this test (Texas Assessment of Academic Skills or TAAS) that was given 
through Spring 2002 in grades 3-8, with an exit examination at grade 10. TAAS was replaced by 
the Texas Assessment of Knowledge and Skills (TAKS). Because the data from TAKS are not 
strictly comparable to TAAS data, they are not considered in this study. 

As shown in Table 20, performance levels for Texas students on all TAAS subtests have 
increased since 1998. TTiis measure (all tests taken) is a composite of reading and mathematics 
scores at all grade levels (3—8 plus exit); writing at grades 4, 8, and exit; and science and social 
studies at grade 8. The data in this table are a composite including all grade levels, and represent 
scores for students included in the accountability ratings. 



Table 20, TAAS Performance for CSR Schools, All Students Statewide, and Economically 



Disadvantaged Students Statewide: Percentage Passing All Tests Taken 



Group 


1998 (%) 


1999 (%) 


2000 (%) 


2001 (%) 


2002 (%) 


CSR 


58.9 


65.2 


80.3 


72.8 


77.7 


Econ disadv 


66.4 


67.5 


70.0 


73.6 


78.2 


All students 


77.7 


78.1 


79.9 


82.1 


85.3 



Source: TEA. AEIS Reports for years 1998-2002 



In Texas, about 28 percent of respondents reported that their schools’ CSR-funded models did 
not continue. (Given that only 23 percent of Texas schools receiving CSR grants returned 
surveys — about half the return rate for other states in this study — it is conceivable that the 
percentage of discontinued CSR models in Texas schools is greater than 28 percent.) One might 
expect that when CSR funding ended in 2000, an immediate impact on the performance of 
students in the schools that had received grants would be apparent. However, just as it is not 
reasonable to expect an immediate upswing in test performance in the first year of a program, it 
is also unreasonable to expect a strong downturn after its final year. It would be impossible to 
reconstruct enough history to substantiate a claim of performance detriments after funding 
ended, although certain results are intriguing. 



The data in Table 20 and in Graph 1 are based on all 137 of the CSR schools, not just those 
responding to the survey. Students in CSR schools made significant performance gains from 
1998 to 1999; however, by 2001, those schools’ performance had dropped to approximately the 
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same level as economically disadvantaged students in the state overall. (In Texas, students in 
CSR schools were approximately 74 percent economically disadvantaged, versus about 50 
percent economically disadvantaged students in all schools statewide.) Graph 1 shows this trend 
more clearly and tracks both the state performance overall and performance for all economically 
disadvantaged students. It shows that student performance in schools with CSR funding declined 
in 2001. 

Graph 1. TAAS Performance, All Tests Taken: CSR Schools, Economically Disadvantaged, and All 
Students in Texas 




The Algebra I end-of-course examination is given whenever a student takes Algebra I. This can 
occur at any grade level from 7 to 12. Not surprisingly, high achieving students take Algebra I in 
middle school. Of the CSR schools in Texas where Algebra I may have been offered, 68 percent 
are middle schools. Results from the Algebra I end-of-course examination are not considered in 
the state’s accountability ratings; however, districts had the option to count the results in 
determining their students’ final grades. 

Table 21 presents information by year for CSR school students, economically disadvantaged 
students, and all students at the state level. These data reflect test information for approximately 
70 schools (because only 18 schools reported Algebra I data in 1998, that year is not included in 
Table 25). As might be expected, the data from the CSR schools (with a high percentage of 
middle schools) indicate a much higher passing percentage than the state overall or for 
economically disadvantaged students. (Data are not reported separately for middle and high 
schools for the state for Algebra I end-of-course examinations.) 




37 



Table 21. Texas Algebra 1 End-of-Course Examination: Percentage Passing 





1999 (%) 


2000 (%) 


2001 (%) 


2002 (%) 


CSR 


57.3 


57.7 


61.5 


66.7 


Econ disadv 


30.2 


31.3 


36.0 


45.1 


All students 


43.4 


43.9 


49.2 


57.8 



Source: TEA AEIS Reports for years 1998-2002 



As may be observed in Graph 2, CSR school students’ initial performance on Algebra I end-of- 
course exams was higher than the state, and it increased at approximately the same rate as the 
state’s overall performance. As the performance levels increase, there is probably a ceiling effect 
that is responsible for decreasing this difference in performance. 



Graph 2. Texas Algebra I End-of-Course Examination: Percentage Passing 




The information in Table 22 is based on approximately 23 schools — primarily high schools 
where advanced courses are offered (The Texas Education Agency lists more than 150 advanced 
courses; for example. Physics II). Data for these few schools are not very stable; still, the CSR 
schools’ student passing rates appear to have increased as compared to rates for economically 
disadvantaged students statewide. By 2001 (the latest year for which these data are available, as 
they are “lagged” one year by TEA), CSR performance was actually slightly higher than for all 
students in the state; however, what is important is not only that percentages are high, but also 
that students are attempting these courses and passing at high rates in schools with many 
economically disadvantaged students. 



Table 22. Texas Percentage of Students Passing Advanced Courses 





1997 (%) 


1998 (%) 


1999 (%) 


2000 (%) 


2001 (%) 


CSR 


17.0 


10.3 


18.4 


19.3 


19.6 


Econ disadv 


13.2 


12.6 


11.3 


13.8 


12.8 


All students 


19.6 


18.9 


17.5 


20.1 


19.3 



Source: TEA AEIS Reports for years 1998-2002 



Two other measures closely related to school completion were examined: student attendance and 
dropout rates. At the student level, an early indicator for dropping out of school is decreased 
attendance. At the school level, attendance rates are highly correlated with grade levels. 
Elementary schools have high attendance rates, whereas rates for high schools are considerably 
less. Middle school attendance rates tend to be more similar to elementary schools, but somewhat 
lower. 

Table 23 and Graph 3 contain attendance data for students in all 137 CSR schools, economically 
disadvantaged students statewide, and the overall state attendance rate. These data, like those for 
end-of-course exams, are “lagged” by one year. In general, attendance rates are at very 
reasonable levels. For example, the required attendance rate to meet adequate yearly progress in 
Texas imder the No Child Left Behind Act is only 90 percent for both elementary and middle 
schools. Table 23 shows a trend of slight increase in attendance rates across the three groups 
since 1997. The trend in attendance rates across the three groups shown in Table 23 is toward 
slight increases since 1997. 



Table 23. Texas Attendance ^ 


lates 




1997 (%) 


1998 (%) 


1999 (%) 


2000 (%) 


2001 (%) 


CSR 


94.1 


94.4 


94.4 


94.7 


94.6 


Econ disadv 


94.8 


95.0 


95.1 


95.4 


95.3 


All students 


95.2 


95.3 


95.4 


95.6 


95.5 



Source: TEA AEIS Reports for years 1998—2002 



BEST COPY AVAILABLE 

er|c 



50 



Graph 3. Texas Attendance Rates 




The other measure related to school completion is the annual dropout rate. In Texas, this measure 
is used as part of the determination of accountability ratings. (Future calculations may reflect 
completion rates — graduation, GED, longitudinal dropout — instead of the annual dropout rate.) 
Table 24 presents annual dropout information. Economically disadvantaged students leave 
school at higher rates than other students; nonetheless, because more middle schools than high 
schools (proportionately) are included in the CSR group than in the statewide “all students” 
group, the lower dropout rate observed for this group is expected. The trend in dropout rates 
shown in Table 24 and in Graph 4 is downward for all three groups. 



Table 24. Texas Annual Dropout Rates 





1997 (%) 


1998(%) 


1999(%) 


2000(%) 


2001(%) 


CSRD 


1.2 


1.2 


1.3 


1.1 


0.9 


Econ disadv 


1.6 


1.6 


1.5 


1.3 


1.0 


All students 


1.6 


1.6 


1.6 


1.3 


1.0 



Source: TEA AEIS Reports for years 1998-2002 



The dropout rate for CSR schools is lower than the state average even though the CSR schools 
have a higher percentage of economically disadvantaged students than the state overall. 
Economically disadvantaged student have a higher dropout rate. On the other hand, because most 
of the CSR schools with published dropout rates are middle schools, the lower rate is expected. 
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Graph 4. Texas Annual Dropout Rates 




Arkansas 

The volume of data available from Arkansas at the school level, although greater than that from 
Louisiana, New Mexico, or Oklahoma, was somewhat limited. Moreover, the data collection 
methods and assessment instruments have undergone changes during the time period of the CSR 
grant and continuing through 2002. Nonetheless, patterns were found to be very similar to those 
observed in Texas. Performance for students in CSR schools either track the state (with higher 
percentages of economically disadvantaged students) or show trend lines that exceed the state. 

Student Performance as Determined from Data Gathered during Site Visits 
The range of educational reform models adopted by schools receiving CSR grants suggests that a 
variety of needs — and, therefore, a range of outcomes — were important to school staff 
Accordingly, desired outcomes and approaches differed among schools implementing the same 
model, whereas similar outcomes were often within the goals of CSR schools implementing 
different models. The absence of a common goal or standardized measure limits this study’s 
capacity to generalize about the overall impact of the CSR program. Planning for school reform, 
to be effective, must be site dependent. Thus the goals and outcomes for Bernalillo Public 
Schools in New Mexico would not (and should not) be identical to those selected by schools in 
Little Rock, Arkansas, or Houston, Texas. 

To be considered appropriate for evaluating the effectiveness of a particular program or model, 
the outcome should match the means for assessing it. While it may seem obvious that desired 
outcomes and assessments should match, with the advent of federal and state requirements, 
changes have occurred that have forced schools participating in CSR programs to concentrate on 
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outcomes different from those originally envisioned and adopted. Of the 15 schools visited, two 
discontinued implementing the model, and five do not implement the model in the same manner 
as when the CSR-fimded model was originally implemented. Most of the changes have occurred 
because the original CSR program did not match the newly required state or federal 
accoimtability requirements. 

An additional factor — ^multiple programs — ^has complicated the issues of assessment and 
outcome. For their CSR programs, several schools implemented more than one model (usually 
two) operating concurrently. It appears that the degree of emphasis on each model has varied 
from time to time, with one model taking precedence over the other as the importance of 
outcomes shifted. (One school that appeared to have implemented four programs at once now 
has none. This is likely a case of taking on too many programs that attempted to address too 
many different outcomes or different aspects of outcomes.) 

Role of Student Performance in Program Continuation 

According to survey results, the most important aspect of a model that led to its continued use 
was successful student outcomes. Survey respondents from schools with continuing models 
selected one or more reasons why their school’s model was continued and indicated the reason 
they considered most important. Table 8 shows that all survey respondents who wrote open- 
ended responses reported that successful student performance measures were the main reason 
that their school’s CSR-fimded model continues to be implemented. One respondent indicated 
that the model helps the school meet its accoimtability growth targets. 

Chapter Highlights 

o Nearly all (98 percent) of respondents in schools where the CSR model is still in place 
noted a positive impact on student achievement. For discontinued programs, 88 percent 
reported a positive impact. 

o School staff have observed several benefits from CSR, many of which are not part of 
formal state or federal accoimtability systems: increased teamwork, increased teacher and 
student morale, increased coordination of teaching, improved teaching strategies, etc. 

o In Texas, students in CSR schools made significant performance gains from 1998 to 
1999; by 2001, those schools’ performance had dropped to approximately the same level 
as economically disadvantaged students in the state overall. 

o In Texas, CSR schools (with a high percentage of middle schools) have a higher Algebra 
I passing percentage rate than the state overall or for economically disadvantaged 
students statewide. 

o The dropout rate for CSR schools is lower than the state average even though the CSR 
schools have a higher percentage of economically disadvantaged students than the state 
overall. Economically disadvantaged student have a higher dropout rate. On the other 
hand, because most of the CSR schools with published dropout rates are middle schools, 
the lower rate is expected. 
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o In Arkansas patterns were found to be very similar to those observed in Texas. 

Performance for students in CSR schools either track the state (with higher percentages 
of economically disadvantaged students) or show trend lines that exceed the state. 
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5 . 

Summary: General Impact and Impressions 



As noted, survey respondents could use either of two similar questionnaires — Siuwey Yes or 
Survey No. A set of general questions was included in both siuwey types (with some variations). 
These questions ask for respondents’ overall opinions about the program’s impact on 
performance, teachers, and parents. They were asked for other opinions such as whether the 
school’s CSR model should be replicated in other school districts and if they thought it would 
continue to be implemented in their school. Table 26 presents the results from Survey Yes 
respondents whose schools are still implementing CSR models. The number of respondents for 
each question is listed in the far right column. 



Table 25. General Opinions Regarding the CSR Program: Model Still in Place (Survey 
Yes) 





Yes 


No 


Don’t 


Total 




(%) 


(%) 


Know (%) 


N 


Do you think this program should be 


94 


3 


4 


80 


implemented in other similar schools? 










In retrospect, do you think this CSR model 


92 


9 


0 


82 


worked well for this school? 










Would you implement this program if you went 


83 


5 


12 


81 


to another school? 










Do you think this program will continue to be 


76 


6 


18 


80 


implemented in this school? 










Do you still use the CSR vendor for support? 


66 


23 


11 


82 


Has this program spread to other schools in the 


64 


25 


11 


81 


district or in nearby districts? 










Do you believe that having a CSR grant had a 










positive effect on: 


98 


2 


0 


82 


Student achievement? 


82 


13 


5 


82 


Parental involvement? 


95 


3 


3 


81 


Professional development? 


95 


0 


5 


82 


Schoolwide reform? 











An examination of the data shown in Table 31 reveals that, overall, respondents have positive 
opinions regarding their schools’ CSR programs. While 94 percent of respondents said that their 
programs should be implemented in other similar schools, about 10 percent fewer said they 
would choose to implement their programs if they went to another school. For this same 
question, another 12 percent said they were unsure, implying that assessing the school situation 
is important before making a decision about whether such a model would be appropriate. Almost 
one in five indicated that they did not know whether their CSR models would continue to be 



implemented in their schools. About one-fourth said that they continue to use their models’ 
vendors for support; roughly the same number believe that use of the models has spread to other 
schools. 

The majority of respondents to Sxirvey Yes believe their CSR programs should be implemented 
in other schools and that the models worked well in their own schools. Respondents indicated 
that the programs positively affected student achievement, parental involvement, professional 
development, and overall schoolwide reform. Fewer respondents) — but still a substantial 
majority (82 percent) — indicated that there was a positive effect on parental involvement. 

Table 27 presents findings from Survey No. These respondents are from schools whose CSR 
programs are no longer being implemented. Overall, their opinions concerning CSR are 
somewhat lower than the opinions of those whose schools are still implementing CSR models. 
More than 90 percent of Survey Yes respondents reported that the programs were successful, 
whereas 71 percent of Sxirvey No respondents believed that the programs were successful. 
Respondents of both survey types, however, said that the CSR programs had a positive impact on 
professional development. 



Table 26. General Opinions Regarding the CSR Program: Model No Longer in Place 
(Survey No) 





Yes 


No 


Don’t 


Total 




(%) 


(%) 


Know(%) 


N(%) 


Was the CSR program successful in this school? 


71 


17 


13 


24 


Do you think this program should be implemented in other 


63 


21 


17 


24 


similar schools? 










Has this program spread to other schools in the district or 


67 


29 


4 


24 


in nearby districts? 










Would you implement this program if you went to another 


58 


13 


29 


24 


school? 










Do you think the model selected for this CSR program was 


58 


29 


13 


24 


the best one? 










Would you implement this program if funding were 


62 


24 


14 


21 


available? 










Do you believe that having a CSR grant has a positive 










effect on: 










Student achievement? 


88 


4 


8 


24 


Parental involvement? 


54 


38 


8 


24 


Professional development? 


91 


4 


4 


23 


Schoolwide reform? 


75 


17 


8 


24 


Did you use the CSR vendor for support? 


68 


14 


18 


22 


If yes, was the vendor or model provider support effective? 


56 


28 


17 


18 



Approximately half (54 percent) of respondents from schools with discontinued models indicated 
that their schools’ CSR programs had a positive effect of parental involvement, and 75 percent 



said there was a positive effect on schoolwide reform. A larger majority felt that their programs 
positively affected student achievement and professional development. 

All respondents were asked to describe the most important changes in their schools that resulted 
from their CSR programs. They listed several positive results for both schools with models still 
in place and those with discontinued models. None of the respondents indicated that negative 
changes have occurred as a result of implementing their CSR programs. 

Schools that are no longer implementing their CSR models reported improvements in several key 
program outcomes, including: student performance, parental involvement, and teacher and staff 
participation and excitement, and quality of professional development. According to one 
respondent, “The program created a greater sense of working together among teachers and staff” 
School organizational changes also took place; for example, a respondent cited scheduling and 
staffing changes at the school. Another said, “Teachers have begun to meet as teams and to 
dialogue more. The school is also more aware of the benefit of providing one-on-one assistance 
to students.” 

Similarly, Survey Yes respondents indicated that several positive changes have taken place as a 
result of their CSR programs. Changes included improved student performance, valuable staff 
development, improved teaching strategies, greater availability of resources and materials, 
increased teacher and school staff morale, and increased parental and community involvement. 
The following are respondents’ comments on outcomes: 

o Teachers are forced to be better teachers, whether they want to or not. 
o During this implementation, district personnel, parents, and students experienced the 
benefits of this research-based program through improved student attendance, grades, test 
scores, and self-confidence. As a result, the district reallocated federal funds to maintain 
the program. 

o Scheduling was impacted because all teachers need to teach reading at the same time of 
the day. More staff members are aware of individual students and their needs. Staff as a 
whole are more aware of the process involved in reading instruction, 
o Program has unified staff, particularly at the primary level, in strategies, goals, and 
cooperation. 

o The greatest impact has been restructuring the math lessons to include the components of 
cooperative learning and higher order thinking, 
o Our focus was on comprehension, thinking skills, problem solving, and student 
interaction. We accomplished these with much success, 
o We have been much more efficient with our staff and the courses we are able to offer 
students. 

o We have a greater number of students reading at or above grade level, and discipline 
referrals have decreased with the increase in reading abilities. 

All respondents, including those whose models are no longer in place, reported that their CSR 
programs were integrated into their school improvement plans. 
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Conclusion 

This study reviewed the implementation status of CSR programs funded in the five states that 
SEDL serves: Arkansas, Louisiana, New Mexico, Oklahoman, and Texas. A total of 268 surveys 
were distributed to all CSR programs funded. The survey return rate was 40 percent. In addition 
to the survey, the evaluation team conducted 1 5 site visits to various schools across the five 
states. Researchers analyzed published student performance data as available, and conducted 
follow-up phone calls with selected schools as required. 

Findings fi-om this study indicate that, overall, the majority of the 268 CSR programs surveyed 
that were originally funded for three years continue to be implemented, some as originally 
implemented, some with modifications. School staff members are satisfied with the models 
selected, and the majority strongly support the programs. Although improved student 
performance is not evident in all CSR schools, those that have continued to use their CSR 
models are committed to them and feel that performance will improve after the models have 
been in place for several years. Aside from improved student performance, other significant 
other benefits were identified that help explain the support and continued implementation of 
CSR models. For many schools, the process of institutionalizing the models has already begun; 
for others, the models have reached the institutionalization stage. 

Despite current funding challenges, many schools have modified their budgets to continue to 
support their CSR models. There was a general consensus among respondents that program start- 
up was expensive; however, ongoing costs decrease after the initial three-year period. 
Nevertheless, for models to be effective, schools still need to provide training, new materials, 
and staff to implement them. While many respondents felt that their school’s model would 
continue indefinitely, some are worried that funding and support for their model could end at any 
time. In some cases, lack of funding has led schools to modify ;their models to. such an extent that 
staff members are concerned about how these changes will affect the overall program delivery. 

General Observations 

Given the specific scope of study and the limitations already described in earlier sections several 
general observations can be derived from this study, these are listed below. 

Student Performance. Student performance, however measured, is considered the singular most 
important outcome that should be considered when gauging the success of these programs. The 
examination of published student performance data did not reveal consistent large increases in 
student performance. However, modest gains were evident. More significant, when student 
performance data were examined during site visits (much of which were not formally 
documented,) outcomes not commonly captured in state reports (like the Academic Excellence 
Indicator Reports in Texas) were much more likely to be noted as a success measure by the 
program administrators. 

Funding. Study findings indicate that the most frequently provided reason for program 
discontinuation was the lack of funding. Survey results and site visits, however, indicate though 
funding is a critical aspect of the CSR program, schools that went through significant changes 
(staffing, teaching strategies, scheduling) as a result of implementing their CSR models found a 
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way to obtain the necessary (if minimal funds) to continue their programs. Regardless of their 
ability to secure continued funding, school staff continue to be concerned that reduced program 
funding might alter the CSR program to such an extend that the benefits of many of these models 
may be compromised by a diminished lack of funding. 

Commitment to Program. Study findings indicate that CSR models are more successful when 
there is a strong commitment by school staff. The program was much less likely to succeed if it 
did not enjoy staff buy-in. To increase and ensure buy-in, schools included staff at an early stage 
of program implementation. Teachers were included as part of management teams. When 
necessary, staffing changes were made to ensure effective model implementation. Some schools 
reported that these programs were so ingrained in the school structure, system, and culture that it 
would be impossible return to a program that did not include the reform model. 

Parental Involvement. While many schools with successful CSR programs indicated that the 
program had improved the communities’ overall perception of the school, many described 
parental involvement as the weakest CSR component. The majority of school staff indicated that 
parental involvement, as related to their CSR program was not producing the desired results. 

Program Evaluation. Like parental involvement, researches found that program evaluation 
activities are weak or lacking in several programs. Importance was focused more on ensuring 
implementation than in program evaluation activities. 
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Appendix A: 

CSR Models Used in the Five States 



4 Blocks of Literacy 
Accelerated Reader 
Accelerated Schools 
America’s Choice 

Advancement Via Individual Determination 
Basic School 
Brazosport Model 
Bridges/Structure of Intellect 
Capacity Building Model 
Cooperative Integrated Reading and 
Comprehensiion 
Coalition of Essential Schools 
Community for Leeiming 
Comprehensive Accelerated Leeiming 
Comprehensive School Mathematics 
Program 
Co-Nect 

Core Knowledge 

Creating Independence Through Student- 
owned Strategies 
Design for Literacy 
Different Ways of Knowing 
Direct Instruction 
Early Literacy Initiative Project 
Education Service Center XX 
Effective Schools 

El Paso Collaborative for Academic 
Excellence 

Every Day Counts Calendar Math 
Expeditionary Learning Outward Bound 



Experiential Service Learning 
Failure Free Reading 
Galaxy Classroom 
Great Expectations 
High Schools That Work 
Higher Order Thinking Skills 
Learning Centered Framework 
Lightspan 
Literacy for All 
Modem Red Schoolhouse 
Motivational Framework for Culturally 
Responsive Training 
National School Conference Institute 
National Writing Project 
Onward to Excellence 
PLATO 

Problem-Based Learning 

Reading and Writing Across the Curriculum 

Reading Recovery 

Roots and Wings 

School Development Program 

Second Generation Effective Schools 

Student Support Teams 

Success for All 

Ventures Initiatives and Focus System 
Voyager Expanded Learning 
Woodlawn Hills Comprehensive School 
Reform Design 




49 



Appendix B: Letters of Support from State Departments of Education 




50 



Appendix C: Site Visit Summaries 



Albuquerque School District, Albuquerque, New Mexico 
Eugene Filed Elementary School 

Austin Independent School District, Austin, Texas 
Regan High School 

Bernalillo School District, Santo Doming, New Mexico 
Santo Domingo Elementary and Middle Schools 

Cypress-Fairbanks Independent School District 
Watkins Middle School 

Edinburg Independent School District, Edinburg, Texas 
North High School 

Fort Bend Independent School District 
Missouri City Middle School 

Jefferson Parish District, Gretna, Louisiana 
George Cox Elementary School 

Jonesboro School District, Jonesboro, Arkansas 
West Elementary School 

Lawton School District, Lawton Oklahoma 
Lawton Elementary School 

Little Rock School District, Little Rock, Arkansas 

Lubbock Independent School District, Lubbock 
Dunbar Junior High School 

Plainview Independent School District, Plainview, Texas 
Houston School 

San Antonio Independent School District, San Antonio, Texas 
Irving Middle School 

Tulsa School District, Tulsa Oklahoma 
Gilcrease Intermediate School 

Ysleta Independent School District, El Paso, Texas 
Yselta Elementary School 
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Eugene Field Elementary School 

Albuquerque School District, Albuquerque, New Mexico 
Initial funding and program implementation 

At the time of the site visit, Eugene Field Elementary School served 430 students in pre K-5 and 
has a schoolwide Title I program. More than 80 percent of students are Hispanic, and all qualify 
for the Free/Reduced Price Lunch Program. The school has English Language Learner and 
bilingual programs; the mobility rate is 126 percent, with several groups of students leaving and 
later returning throughout the year. The school is on probationary status, as determined by the 
New Mexico State Department of Education. In 1999 Eugene Field received $50,000 per year of 
CSR funding to implement the Design for Literacy (DFL) reform model. 

A team of teachers, administrators, and parents began with the idea that the school needed 
additional funding to improve its reading program. In addition, a new district policy encouraged 
all schools to adopt a literacy model. Teams of teachers visited area schools implementing 
various reform models. Assisted by a consultant from the state education department, they 
selected Design for Literacy (DFL) because it was comprehensive and dealt with accelerated 
reading, remediation, intervention, and prevention. DFL’s extensive professional development 
program includes classroom support for the teaching of literacy; appropriate instructional 
materials; and uninterrupted time for literacy teaching and learning. The program also includes 
strands for family and community involvement, school governance, leadership, and intervention. 
Teachers felt that DFL would give the school opportunities to create small learning groups of 
students and would address the needs of second language learners. The principal engaged 
university faculty and retired Title I teachers to help write the CSR grant application. Prior to 
implementation, all teachers went through intensive training. Teachers who did not want to be 
part of the reform were allowed to transfer to another school. 

The school instructional council made all CSR funding decisions. During the first two years of 
the program, funds were used for professional development, classroom materials, and DFL 
library books. The district provided an instructional literacy coach. 

The post-CSR program 

Implementing DFL has improved instruction. By the third year, scores for every grade level had 
improved in all areas on standardized tests. Reading scores improved significantly. The school 
continues to implement and expand DFL (a similar program is planned for math); however, the 
program is at risk, based on current funding levels. After initial implementation, the cost of 
maintaining DFL decreased, but funds are still required for staff development and stipends. The 
district is supportive but unable to provide additional funds. The school has used Title I, 
operational, and School Improvement funds to support the program and has refurbished books 
and other materials needed for the program. Student/teacher ratios, small during the funding 
period, have increased. 

Before CSR, teachers used a variety of instructional approaches. With DFL, there is more 
coordination in instruction, and more resources are available for literacy programs. Teacher 
turnover has been reduced. Teachers like the structure of the model, are excited about their 
students’ accomplishments, and believe that they are making a difference. Instructional coaches 
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model teaching strategies, provide assistance, and conduct weekly training sessions. The school 
no longer uses external training consultants. 

CSR funding — and DFL — ^have positively affected both students and teachers at the school, as 
well as the larger commimity. The program has helped the school establish a comprehensive, 
state-of-the-art literacy library. Eugene Field has achieved a reputation in the community for 
being effective. Previously, the school did not attract students or good teachers. Now, parents 
want their children to attend and teachers want to join its faculty. 
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Reagan High School 

Austin Independent School District, Austin, Texas 
Initial funding and program implementation 

Reagan High School serves 1,300 students in grades 9-12: half Hispanic; 40 percent, African 
American. One-third of students are economically disadvantaged, and about one-fifth limited 
English proficient. The mobility rate is 33 percent. Reagan has a schoolwide Title I program. The 
school received $59,700 per year of CSR funding starting in 1999 to implement the 
Advancement Via Individual Determination (AVID) model. That year, Reagan was rated 
“Acceptable” in Texas’ accountability system. 

The district applied for and received CSR funds to implement AVID in four high schools and 
four middle schools. AVID serves students with good grades in regular classes who show 
willingness to take rigorous course work and potential for graduating from high school and 
attending a university — frequently the first in their family to do so. AVID engages students as 
freshmen and continues to work with them through their senior year, focusing on collaboration 
and on writing, organization, and thinking skills. Students and parents sign contracts agreeing to 
the program’s rigorous requirements. An initial challenge was selecting students who would 
benefit from participation in AVID. Over time, the school has fine-tuned selection criteria. 

The AVID site team met monthly; the district AVID coordinator was responsible for routine 
decisions. CSR funds were used for teacher salaries, stipends, and staff development, as well as 
to establish AVID classrooms and furnish them with computers and printers. Program funds also 
purchased hardware, software, and materials for AVID students, and were used for field trips, 
primarily to area colleges and universities. 

The post-CSR program 

Although participating in CSR was not initially a campus-level decision, the Reagan staff now 
fully supports the AVID program. When CSR funding ended, the district worked with the Dell 
Family Foundation to secure additional funding to expand the program in all district high 
schools. Title V and Title I funds are also being used to maintain AVID. Reagan’s principal is a 
strong and vocal supporter of the program. 

Because AVID encourages students to enroll in honors and AP classes and helps them graduate 
and prepare for college, its goals align with overall campus, district, state, and federal goals. Part 
of its No Child Left Behind (NCLB) funds will be used to fund the AVID program. In addition, 
it appears that the district’s Gear Up initiative intends to apply AVID principles in order to 
increase performance scores and to align with NCLB. 

Though AVID has continued at Reagan, its operating budget has been significantly reduced, 
mainly for teacher training and student field trips. Cxirtailing field trips is of particular concern 
for these students, many of whom would not otherwise be able to visit universities. Nonetheless, 
Reagan’s AVID program has become the largest in the district. AVID strategies are beginning to 
be used in all classes. 



AVID has successfully increased the number of students taking reading and AP classes, and 
university entrance exams. Increasing numbers of students have enrolled in college. This year, 21 
students from the first cohort who started AVID in grade 8 will graduate. All but one will attend 
universities. Reagan’s financial aid office has helped to link them to financial aid systems 

Teachers apply for AVID positions, which they consider rewarding because they teach 
motivated, college-bound students. They become surrogate parents and individual counselors, 
often working one-on-one with students. Still, with teacher turnover, it has been hard to maintain 
a professional knowledge base, and teacher training has suffered from lack of funding. While all 
teachers previously attended yearly training sessions, now only new teachers are now being 
trained by campus AVID teachers. This year Reagan has established Saturday training sessions 
to allow AVID teachers to coordinate and help each other develop lesson plans. 
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Santo Domingo Elementary and Middle Schools 
Bernalillo School District, Santo Domingo, New Mexico 

Initial funding and program implementation 

The Santo Domingo Elementary School, located within the Santo Domingo Tribal Nation near 
Albuquerque, serves 350 students in grades 1-5. About 170 students attend the adjacent grade 
6-8 middle school. One principal is responsible for both campuses. Santo Domingo Elementary, 
which has a schoolwide Title I program, was placed on probation by the state department of 
education in 1999. 

The Title I director initiated the process for CSR funding, and the district paid for substitutes 
while teams of Santo Domingo teachers visited area schools serving students with similar 
demographic characteristics to select a model. They chose Success for All (SFA) for its emphasis 
on accelerated reading, remediation, intervention, and prevention. Subsequently, the principal 
pushed for its adoption. After much negotiation, 80 percent of teachers agreed; however, during 
the first two years, approximately 35 teachers left the school, due to dislike of the program, 
unwillingness to alter their teaching approach, or, according to the principal, a mindset that “our 
students — ^Native American, rural, second language learners — ^needed lower academic 
expectations.” 

Most CSR funds were spent for materials and, initially, for training. During the first two years, 
SFA was implemented without modifications. In the third year, the school “customized” the 
program: ESL components were introduced and SFA groups were established in the middle 
school. 

The post-CSR program 

SFA has not been diminished as a result of the grant ending — in fact, the program has grown and 
expanded. It has become so thoroughly integrated into the instructional delivery and culture in 
both schools that when CSR ended, discontinuing SFA was “not an option.” The district 
continues to fund SFA facilitators. Additional basal sets were purchased so that students 
repeating SFA levels could read fresh materials, and Native American and local literature 
sources have been incorporated into libraries. A variety of funding sources currently provide the 
approximately $20,000 per year needed to operate SFA: district operating funds and incentives. 
Title I and Title VIII fimds, and a 21st Century grant. The elementary school has also applied for 
the NCLB Reading First grant. 

The elementary school currently remains on probation (and has been placed on Corrective 
Action); and program and school goals have not been met (100 percent of teachers proficient in 
SFA techniques and target increases in student performance on standardized tests). Still, 
interviewees agreed that the school’s previously fragmented reading program has become 
stronger as a result of implementing SFA. The program has also helped teachers define and focus 
on reading and has enhanced cooperation among teachers. Students, now progressing at their 
own pace at appropriate reading levels, have also benefited. Third graders are currently the 
strongest class in reading (these students have been instructed using SFA techniques since 
kindergarten), and student interest in reading has increased. Daily tutoring sessions are available 
to all students. 
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Watkins Middle School 

Cypress-Fairbanks Independent School District, Houston, Texas 
Initial funding and program implementation 

Watkins Middle School serves 1,150 students in grades 6-8 in urban Houston. After notifying 
Watkins of eligibility for CSR, the district provided little additional assistance. In 1999, Watkins 
began receiving $109,648 per year of CSR funding for three years. 

All educational decisions at Watkins are based on its three-part plan, which provided the 
framework for the selection of Creating Independence through Student-owned Strategies 
(CRISS) and Advancement Via Individual Determination (AVID). The CSR committee 
researched models and examined results from schools with similar demographics. CRISS was 
chosen for its campus-wide focus and its potential for helping low SES and Hispanic students 
with writing and with test taking skills, and for its parent involvement component. Final selection 
of CRISS and AVID was made after input from the entire faculty, parents, and students. Student 
survey results led to the inclusion of two additional programs — Book It to the Library and a 
computer lab program — which allowed students to work on computer skills or conduct library 
research after school, supervised by staff members paid with CSR funds. 

CRISS, a schoolwide generic reading comprehension program, is applied across content areas. 
AVID is a college preparatory elective that utilizes group tutorials facilitated by university 
students. A secretary, paid by the grant, kept records and accounts. Program decisions were made 
by the CSR committee. Initial expenses were related to teacher training, purchasing materials for 
implementing CRISS strategies, and setting up AVID classrooms. The district funded half (and 
later, all) the salary of an AVID teacher. First-year training for CSR programs included in- 
service sessions and follow-up activities such as lesson plan checks and classroom observations. 
Training continues on an ongoing basis, with experienced teachers training new ones. Teachers 
are encouraged to attend district workshops and state conferences related to CRISS and AVID. 

The post-CSR program 

Interest in the after-school library and computer programs waned after two years, and the 
committee deemed them no longer needed. Within three years, however, CRISS had become part 
of the regular school curriculum; teachers incorporate CRISS strategies into daily lessons, and 
these strategies have become part of students’ daily routines. Costs have been “absorbed into 
school expenses.” Three teachers have become national-level trainers: one for CRISS and two 
for AVID. Both programs are regarded as “strong” academically, but AVID is “struggling” with 
funding through snack sales, sponsored events, and donations. Watkins hopes to continue AVID 
because it provides opportunities not otherwise available to students. AVID students report that 
they have formed “family” bonds with each other, with their tutors from local universities, and 
with their teachers. Standardized test scores have improved, dropout rates remain low, and more 
minority students now take advanced classes. It is expected that the district will continue paying 
one FTE in support of AVID. Otherwise, AVID has been “cut to the bare minimum.” This year, 
Watkins received support from NCLB after-school funds for tutoring; next year, however, there 
will be no NCLB after-school initiative. The district has not supported the program well since 
CSR funding ended, being “too far removed from the classroom to understand” the changes in 
the student population. Because funding is tight, the two classes per grade level of AVID may be 
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reduced to one class. In addition, the college student tutors may be dropped and required teachers 
to take over tutoring tasks during their conference times. 

CSR-funded programs have contributed to an atmosphere change at Watkins, as the school 
moved from a punishment-centered approach to discipline to a “cohesive, caring family” 
environment for students. Teachers’ attitudes toward students have changed, communication 
with parents has improved, and a sense of partnership has developed. Parent nights are viewed as 
very effective for encouraging participation of traditionally reluctant groups. After positive 
experiences, many parents have begun to volunteer at Watkins. Selection of Watkins as an AVID 
national demonstration school is attributed to the cohesive and effective functioning of the site 
team. This honor, plus inclusion in the Texas Mentor School network, led Watkins to apply for 
the Blue Ribbon Schools Program. 
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North High School 

Edinburg Independent School District, Edinburg, Texas 
Initial funding and program implementation 

North High School, located in the Texas Rio Grande Valley region, serves 1,900 students in 
grades 9-12, most from economically disadvantaged families. The Charles A. Dana Center at the 
University of Texas at Austin came to the district and presented the AVID program and its 
merits. North High School applied for the CSR grant. The area coordinator for AVID, based at 
Pan American University in nearby McAllen, has supported AVID at North and maintains 
contact with the Dana Center. 

Prior to CSR, North’s special programs concentrated on AP coursework and on recovery 
strategies for struggling students; little was offered for the average student. Although 
interviewees emphasized that AVID was for “the middle student . . . who can do better and can 
go to college,” the principal characterized all AVID students as at-risk. Recruiting students is 
problematic because teachers, students, and counselors do not understand eligibility criteria. 
AVID is rigorous and demands more from students than the regular school program requires. 

The approximately 70 students selected for each grade level are encouraged to take AP courses 
and university entrance exams. 

Staffing constituted the main cost for AVID. First-year implementation costs were highest 
because rooms were changed and furniture bought. Although second-year costs were lower, there 
were maintenance expenses for technology, networks, and printers, as well as expenses for 
training the staff and for Dana Center consultants, who provided annual training and one summer 
workshop during the three grant-funded years. The school paid to send staff members to Austin 
for training. The Intel Corporation donated computers and training, and Dell provided a grant to 
purchase laptops. Expenditure categories were shifted to buy computers for students’ use. 

The program post-CSR 

Although the principal said that the program has not suffered from lack of grant funds, 
coordinators said that AVID is on “bare bones” and losing momentum; moreover, the absence of 
standardized test score increases has cast doubt on its continuation. This year, the AVID teacher 
who was interviewed donated some of her own time for AVID. She and others who are involved 
do not feel compensated for time spent on paperwork and after hours work. Because all AVID 
students qualify as at-risk. Compensatory Education funds are available, and perhaps will pay for 
the program if it continues. The principal added that current funding for AVID is now as an at- 
risk program and is coordinated with general AP programs. Gear Up, a schoolwide initiative for 
all students that is similar to AVID, has also been implemented at North. While the goals for the 
two programs are similar, the difference is that AVID is student-centered, is highly selective, and 
has more depth than the Gear Up program. It appears that AVID is competing for funds with 
Gear Up at the district level. 

AVID has increased the number of North students enrolled in reading and AP classes. All AVID 
students graduate. Nearly three-quarters of North students have taken the university entrance 
exams — ^the most in the region. In addition, a greater percentage of students use computers as a 
result of AVID. AVID teachers have become mediators and advocates for students, who now 
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gain more recognition in other classes. On the other hand, schoolwide support is lacking. The 
program remains isolated and misunderstood. There is a resistance to change teaching styles, and 
non-AVID faculty members mistakenly perceive AVID as an at-risk student program. Training 
more teachers in order to make AVID a genuine schoolwide program was suggested as an 
improvement that AVID should make. 
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Missouri City Middle School 

Fort Bend Independent School District, Sugar Land, Texas 
Initial funding and program implementation 

Missouri City Middle School (MCMS) serves 1,200 students in grades 6-8 and has a schoolwide 
Title I program. MCMS received $85,740.00 per year starting in 1999 to implement four 
instructional models: Second Generation Effective Schools, Mastery in Learning, Second Chance 
Reading, and Just Read! The district administrator responsible for involving MCMS in the CSR 
program and selecting its models regularly visited MCMS to train teachers in their use. District- 
level staff development was also available. 

The post-CSR program 

Since the grant ended, no additional funds have been available to continue components of the 
models beyond what is supported within basic district and/or campus budgets. The most 
noticeable change when the grant ended was the curtailment of local summer institutes. These 
two-week institutes had provided intensive reading instruction for recommended students, as 
well as hands-on training for teachers, under the guidance of model providers. 

The Just Read! model required paperwork that both students and teachers found overwhelming. 

In addition, some of the activities — such as book reports and proofreading of drafts — were 
beyond the capabilities and the interests of many students. Moreover, all the models selected for 
CSR funding were intended to target low socioeconomic status (SES) students — particularly ■ 
those living in homes with someone other than their parents. Unfortunately, the parent 
involvement component seemed to attract only those parents who were already involved with the 
school and failed to attract the others. Efforts to keep components of the CSR-funded models 
“pure” and intact have been hindered by teacher turnover. In addition, the advent of the new state 
assessment system has distracted attention away from these models; most professional 
development is currently focused on what will enhance performance on these tests. 

The original aim of the two reading models funded by CSR was to increase reading among 
students who had no interest in reading. Books with high interest and low reading level, 
addressing different cultures, were purchased with grant funds. These books, described by the 
principal as “phenomenal,” are specially tagged and available to students through the library. 
Both the reading instruction and classroom management components of the CSR-funded models 
are considered beneficial for all students, with the possible exception of those functioning at the 
highest levels. Interviewees noted that many students had increased their vocabulary 
dramatically. In addition, cooperative learning groups have provided opportunities for students to 
become “mini instructional leaders — ^to take ownership of the learning and to go beyond what 
they’d normally do in a classroom.” 

As a result of CSR, there is more consistency in instructional strategies and higher level of 
awareness of campus goals. Teachers benefited from the “top-notch training opportunities” built 
into the CSR-funded models. Since the grant ended, teachers have trained each other. MCMS 
will receive a NCLB grant. Funds will be used to implement the AVID model and, if possible, 
the CRISS model. These models are expected to serve the high-poverty MCMS student 
population better than models implemented with CSR funding. Although interest has lagged 
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since the departure of the central office administrator who initiated the grant request and the 
librarian who was “cheerleader” for the program, strategies and competencies inherent in the 
CSR-funded models considered effective in improving student achievement will continue to be 
used in MCMS classrooms, in more or less attenuated forms. Several instructional strategies 
initiated as part of CSR — in particular, cooperative learning, pair-share, peer tutoring, read- 
aloud, and Drop Everything and Read — have been institutionalized. They were either 
incorporated into the MCMS Campus Strategic Plan, or they appear in lesson plans and in 
classrooms. 
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George Cox Elementary School 
Jefferson Parish District, Gretna, Louisiana 

Initial funding and program implementation 

George Cox Elementary School serves 600 K-6 students in a community consisting primarily of 
apartment and temporary housing dwellers — ^nearly all are from low SES African American 
families; the school’s mobility rate is 50 percent. The parish Title I office advised Cox to apply 
for CSR, with the directive to raise test scores. Otherwise, the program was implemented with 
minimal parish support. In 1998, three parish schools received CSR funding. The parish favored 
Co-Nect’s technological aspects, and Cox followed suit for consistency — even though its 
principal preferred “other ways of knowing.” Interviewees were favorably impressed with Co- 
Nect’s Problem-Based Learning (PBL) curriculum because it encompassed a wide range of 
student abilities — important at Cox, where student characteristics drive the pedagogical approach 
toward a holistic and student-centered focus; incoming students had little or no readiness for 
grade level work, and homework was not a cultural expectation. Another CSR initiative, started 
at Cox in 1999, is HOSTS (Help One Student to Succeed). 

The $50,000 from CSR paid for the Co-Nect license and for the consultant. Title I funds paid for 
three years of the coordinator’s salary and all travel for staff development. Eventually, Co-Nect 
entailed additional expenses for its Critical Friends (CF) component. For training, teachers 
attended Co-Nect and CF conferences and a technology mini-sabbatical. A traveling computer 
lab was available, and training by the consultant. 

Implementation was fraught with problems. First, teachers found it hard to create their own 
curriculum without clear structure. PBL uses a holistic approach that not all teachers are ready 
for; without textbooks they became insecure and nervous about their students’ test performance. 
High turnover among teachers also caused problems. The coordinator said she felt ready for PBL 
by the time funding ended. Nonetheless, some projects described as “fabulous” resulted from 
teachers’ and students’ efforts. 

The program post-CSR 

Before CSR, Cox had participated in Accelerated Schools (AS), restructuring into four wings to 
create schools within schools. The PBL curriculum functioned well within the wings. Previously 
implementing AS had helped Cox to be ready for PBL. Most interviewees agreed that what could 
be considered “comprehensive” about CSR in their school was attributable to the work 
previously accomplished with AS. 

Parish goals and the new Louisiana Educational Accountability (LEAP) system do not align 
directly with Co-Nect, yet Cox earned the “recognized” growth status for Louisiana’s 
accountability system by achieving a significant increase in student performance on LEAP 
reading tests. Interviewees ascribe success more to AS than to Co-Nect. Recently, the NCLB 
initiative has driven the state’s educational impetus and curriculum. The emphasis on testing 
creates conflict for teachers who are trying to implement PBL. 
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Although Title I will pay for one more year of Co-Nect, interviewees saw little benefit in this 
expenditure. They liked Co-Nect, but they felt that too much emphasis was placed on 
technology. PBL has been institutionalized at Cox, to the extent that those trained in PBL are 
encouraged to apply it in the classroom and to adapt it to LEAP. However, high teacher turnover 
rates, parish-level budget reductions, lack of teacher engagement, readiness challenges of the 
student population, and conflict with the new accountability system have weakened Cox’s 
commitment to Co-Nect/PBL. Still, interviewees would like to continue training and articulating 
PBL with the LEAP scope and sequence. Referring to LEAP’S structured 10-year benchmark 
process, they agree that comprehensive programs such as CSR represent the only way to 
effectively serve their students. 

After five years, HOSTS will remain in place (the 10 lowest performing schools in the parish 
were eligible for funding). HOSTS’ success in enhancing student performance is a result of its 
assessment and diagnostics components, which are integrated into a student’s individual 
educational plan and used to structure formal one-on-one tutoring. HOSTS, which currently 
serves 35 students, brings community mentors into the school every day. 

Interviewees noted that because of the CSR effort, more students were interested in coming to 
school and learning, and that grant-funded programs were clearly benefiting students’ 
performance, although data could not show a direct correlation between the program and 
academic performance. By contrast, the coordinator said that in hindsight, the $50,000 of the 
CSR funding was wasted, and that PBL was the only thing they have now without major 
expense. She felt that Co-Nect had misled the school by offering technology as panacea. External 
technical support was ineffective because teachers are not expert users. It was recommended that 
a network supervisor be part of the program for assistance and supervision in the lab and for 
repair and maintenance of hardware and printers. (Note: The Success for All programs selected 
the school as a pilot site and offered the program at no cost. Interviewees said that they were 
disappointed after their Co-Nect experience and tired of struggling to implement LEAP scope 
and sequence; thus they refused the opportunity.) 
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West Elementary School 

Jonesboro School District, Jonesboro, Arkansas 
Initial funding and program implementation 

West Elementary School serves 400 students in grades 1-5 and has a schoolwide Title I program. 
The initial decision to participate in CSR was made by the West School Improvement Team and 
the Title I committee, working with the district-level program director to make sure that CSR 
would merge with other priorities and funding sources. The traditional approach was no longer 
working with West’s student population, which was “changing drastically every year.” After 
notification of the opportunity from the state department of education, a group from West 
attended a CSR orientation, where they identified the extended language arts block and parent 
component in Success for All (SFA) as appropriate for their students. Administrators did not 
attempt to “sell” SFA, recognizing that such a rigorous program would fail unless it had strong 
buy-in. 

West received $50,000 per year starting in 1999 to implement SFA. SFA is expensive — ^the total 
cost for the initial year was close to $250,000 — ^most of the $50,000 from CSR going toward the 
SFA contract, which includes the intensive on-site training. To supplement, the school used 
district. Poverty Index Program, and Title I funds. The district contributed a certified 
teacher/tutor. After the first year, CSR also helped pay for components of Lightspan math, and 
for Accelerated Reader and Math. 

The SFA training sessions have been considered very helpful, as are the periodic external 
assessments of reading progress used for regrouping students. Still, some teachers initially did 
not like the scripted nature of the model; and the intensive pacing has been difficult to maintain 
for its 90-minute duration — after which all non-reading subjects must be “literally crammed” 
into the rest of the day. Another difficulty is that when students are placed at their appropriate 
level and follow the SFA curriculum for that level, they may not be exposed to material that 
appears on the state assessment for their grade. Finally, the fact that West has implemented a 
primary reading program other than the one adopted by the state has created difficulty for the 
school within the district. 

The post-CSR program 

After its third year, although test scores had not yet improved, teachers chose to continue with 
SFA because they believed it benefited students and they wanted to follow the original group 
through grade 5. Moreover, SFA materials, procedures, and staff members were in place, and 
there was no clear vision of a replacement for SFA. Ultimately, scores improved, and the 
program remains in place. 

Programs have not suffered as a result of the CSR grant ending, due to skillful forecasting about 
what would be needed when the funds ran out, availability of Title I funds, help from the district, 
and lower annual program costs. Because LightSpan, Accelerated Reading, and Accelerated 
Math are relatively inexpensive, and because their use has become fairly standard, they will 
probably continue. However, paying for SFA will continue to be a challenge. Current funding 
comes mostly from federal entitlements. In addition. West will receive funding for the NCLB 
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after-school tutoring program. Even if not funded in future, the basic training and SFA strategies 
are ingrained enough that West teachers could continue using them without external support. 

SFA is described as having “changed the entire school” by giving teachers a common vocabulary 
and approach, and encouraging them to regard students as “our kids.” They work as 
teams — much as they have taught students to work in cooperative learning groups — and have 
developed support systems for each other. SFA assessment information available at regular 
intervals has provided a useful gauge of students’ growth in reading. Because many come from 
chaotic home environments. West’s students thrive on SFA’s structure. The cooperative learning 
groups help to fulfill their need to belong, and built-in reinforcement, celebration, and conflict 
resolution are also important components. SFA consultants recognize the shift toward the state 
accountability system and have offered related sessions at their conferences, focusing more on 
student performance than on strict program implementation. West has also created documents 
aligning SFA with district benchmarks and wiA testing program. Positive achievement gains 
appeared this year. Some second and third grade classes had averages on the SAT 9 in the top 
quartile, for the first time. Although West’s scores on the state assessment were still the lowest in 
the district, achievement gaps were smaller. 
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Lawton Elementary School 

Lawton School District, Lawton, Oklahoma 

Initial funding and program implementation 

Lawton Elementary School serves 220 K-6 students — ^nearly all at poverty level. Lawton was 
rated low-performing on the basis of test scores and is eligible for schoolwide Title I funding. A 
state-level Title I contact invited Lawton to a presentation by CSR model providers and 
encouraged the school to consider adopting SFA. The principal and a teacher visited other 
schools implementing SFA programs and wrote the application. 

Prior to CSR, teachers had worked separately, and the school environment lacked cohesiveness 
or a systematic approach. Interviewees were very positive about the schoolwide elements of 
SFA. They described SFA’s comprehensive training as outstanding and commented on its 
excellent, individualized service. They agreed that the “user-fiiendly” providers were committed 
to serving Lawton. The state held yearly conferences for SFA schools to meet and share best 
practices. Although initial district guidance was minimal, support increased as Lawton 
demonstrated success. Interviewees said that SFA is so well organized and self-contained that it 
was fairly easy to implement. Initially the program had a “core” of supporters, while some 
teachers were resistant to change — many not liking the close interaction with students and 
parents. An additional challenge, teacher turnover, affected program continuity. It was suggested 
that the program should have some structure such as peer training to deal with this issue. 

The coordinator characterized the CSR grants as a “lifesaver” because it paid for everything 
needed in the classroom as SFA was implemented. The grant money also helped expand the 
library. Costs remained the same for the three years of the CSR grant except for training, which 
was more expensive during the first year. Title I funds supplemented personnel costs and paid for 
materials. Other general funds paid for materials and conference attendance. Title I and state 
funds supported supplemental expenses. The school’s high percentage of students from low 
income families qualifies it for additional support from the district. 

The program post-CSR 

The most important outcome for CSR was the harmony among all partners in 
education — community, school, and teachers — ^with a student-centered purpose. “The students 
have become our children and the school has become our school.” School personnel now “speak 
the same language,” and the school has become a community of learners, with educators working 
as a team. Children are better behaved. 

Lawton’s Academic Performance Index has shown gains, and student performance has improved 
as measured by SRA’s eight- week assessments. Scholastic Reading Inventory, and CRT for 
grade 5. The ITBS scores for language arts have also risen since first year of implementation. 
There are concerns about math scores, but steps have been taken to improve those and the SFA 
math program. In addition, efforts focused on improving students’ test taking strategies have 
been successful. Special education students were most positively impacted; still, all students 
benefited. 
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Campus educators chose to continue SFA for a fourth year with Title I funds because of its 
success in performance and in improving student behavior and parental involvement. This 
success has created much support for the program from parents and teachers. The school’s 
poverty status has changed because higher income families are attracted by its good reputation. 
Lawton had been targeted to close by 2006, but now Lawton is used as a model for other schools 
in the district. 

Nevertheless, next year the school must follow the district alignment to standardized assessment 
with the RF program (the district did not give the school an option to select SFA; they would 
have preferred that SFA funding last at least six years). Lawton’s principal and program 
coordinator noted that if RF does not produce results as positive as SFA, they will find a way to 
reinstate SFA. SFA has been internalized and “no matter what,” teachers are determined to 
continue training, scheduling, tracking data, and providing family support. They have the 
training and the structure for SFA, and they are committed to keep the program going. For year 
five, no funds are available for SFA contracts or materials. To save on expenditures, teachers 
have begun to reuse materials. SFA providers continue to consult at no cost on an informal basis, 
and the school will continue peer training to “build on what we have achieved” and ensure the 
continuity of SFA at Lawton. 
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David O. Dodd Elementary School 

Little Rock School District, Little Rock, Arkansas 

Initial funding and program implementation 

Dodd Elementary School, on the outskirts of Little Rock, serves 212 preK-5 students and has a 
schoolwide Title I program. Dodd initiated its CSR application with assistance from the 
University of Arkansas-Little Rock (UALR), the Arkansas State Department of Education, and 
SEDL. Dodd received $50,000 per year starting in 1999 to implement the Reading Recovery 
(RR) model and the Futurekids model for technology enhancement. 

Reading Recovery was implemented in combination with Early Literacy Learning in Arkansas 
(ELLA), a spin-off of RR utilizing “levelized” books, small groupings of students within 
classrooms, and a campus-based literacy coach working with teachers. To adopt that model, the 
principal had to contend with the district’s strong support for a particular basal series and for the 
Success for All model. Futurekids, a technology-oriented literacy program, was also 
implemented, but proved too difficult for Dodd students. The program was dropped after the first 
year and replaced by an intermediate grade program utilizing the AlphaSmart word processing 
system. 

Initially, the RR teacher and the media specialist made most decisions related to CSR funds. 

After the literacy coach joined the team, she assumed leadership. Teachers contributed to all 
purchasing and other decisions. Throughout the grant period, the most money was spent on 
materials, less on professional development and training, and the least on equipment. Title I 
funds have paid the RR teacher’s salary, whereas CSR funds paid for teacher training, including 
conferences and courses. Grant funds spent for grades K-2 were related to changes in the 
classroom environment needed for the new literacy program: furniture, books, and materials; 
whereas purchases for grades 3-5 included AlphaSmart word processors, software, projectors, 
and other equipment. During the first year, funds also went toward staff development for the 
Futurekids program. After Futurekids was dropped, a consultant provided training for 
AlphaSmart, modeled teaching practices, worked with students, and assisted teachers in 
designing classroom activities coordinating literacy and technology. 

The post-CSR program 

CSR enabled Dodd to make a comprehensive reform of it entire school: “There is nothing that 
hasn’t changed.” The entire classroom setup is different, as well as the delivery of instruction. 
Levelized reading materials are centrally located and easily accessible. Discipline problems have 
dramatically decreased. Test scores have improved for first and second grade students, and 
students’ writing skills have greatly improved. Since implementing the AlphaSmart system, 

Dodd educators consider their school to be on the “cutting edge.” The technology program has 
helped to create a “community of learners; kids see teachers learning, and they see that learning 
is a lifelong activity.” Annual Technology Nights provide opportunities for students to 
demonstrate to their parents the technology they use routinely. The number of parents who attend 
the school events continues to increase. 

Now, all Arkansas schools must implement ELLA, which, combined with RR and the 
AlphaSmart program, aligns closely with the state, campus, and Title I goals. Dodd’s programs 
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have not yet suffered as a result of the grant ending — despite the fact that Dodd is a poor school. 
However, over time, the programs may not be completely sustainable. Three Dodd teachers have 
been trained in RR methods. With the literacy coach, they will continue to provide the 
professional development associated with literacy at Dodd via informal lunchtime and after- 
school meetings that teachers attend voluntarily, and by modeling with students in teachers’ 
classrooms. The district assisted Dodd in obtaining a NCLB Reading First grant, which will fund 
the literacy coach’s position for three more years. Meanwhile, to pay for needed 
materials — particularly for the AlphaSmart systems and the consumables needed for the literacy 
program — ^beyond what it can afford with the Title I funds allocated annually (about $500), 

Dodd holds fundraisers and brings in additional money through campus sales of snacks. Funds 
from those sources will probably not be sufficient to maintain the current number of AlphaSmart 
systems over time. Moreover, funding for the literacy coach probably will not be available after 
the three years of NCLB funding end. 



Dunbar Junior High School 

Lubbock Independent School District, Lubbock, Texas 
Initial funding and program implementation 

Midsize Dunbar Junior High School has changed in character due to resetting of boundary lines 
in the district. More than one-fourth of its grade 7-9 students are in special education. Most are 
classified as economically disadvantaged. The recent influx of students from a school that closed 
was evidenced by lower passing rates on the state exams and by increased referrals to 
disciplinary alternative education programs. 

Dunbar initiated the CSR application after its principal, who was interested in moving beyond 
testing requirements to higher levels of student achievement, learned of the grant at a conference. 
The Charles A. Dana Center at University of Texas at Austin assisted with the model selection, 
whereas The Texas Education Agency provided support in submitting the grant proposal. Dunbar 
received $122,000 per year starting in 1998 to implement the Advancement Via Individual 
Determination (AVID) program. Dunbar staff members visited schools implementing the AVID 
program, and the campus AVID leadership team attended national training. Staff members both 
at the district and campus have other professional ties to AVID (either by serving on the AVID 
advisory board or by providing training). Given their level of commitment, AVID support of 
Dunbar’s program is assumed to be very high. 

The principal, with the involvement of the campus AVID leadership team, determined how CSR 
funds were spent: they supported an AVID teacher, program costs, instructional supplies, a 
counselor (partially fimded by AVID), and travel for staff and students. Early in the program, 
some funds were used to purchase computers. 

The post-CSR program -t. • • 

Interviewees were not only pleased with the program’s impact, but also strongly support and 
advocate for the program. The decision to continue with AVID was never in doubt. Title I funds 
now support Dunbar’s AVID program, and, thus far, little has changed in its functioning. All 
teachers on the campus have been trained in AVID approaches. Although overall district 
endorsement does not appear to be present, the presence of the principal and district coordinator 
(now associated with AVID, as previously noted) will likely ensure that financial and 
professional effort required to sustain the program will continue. 

Because AVID has been vertically aligned across Dunbar’s feeder elementary school and its 
receiving high school, elementary students are becoming familiar with AVID concepts and 
former AVID students are supported in the high school. On the other hand, maintaining parent 
involvement has proven difficult since the integration of students from the closed school. 
Moreover, the program at Dunbar faces difficulties in the next year because the lead AVID 
teacher/campus coordinator will be leaving. 

Examination of expenditures over time shows relatively constant spending within categories 
such as travel and salaries, whereas spending for technology declined significantly. Dunbar 
receives Ninth Grade Success Initiative (NGSI) funding from the state, and AVID is also 
coordinated with this program. It is assumed that because Dunbar achieved a “recognized” status 
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in the third year of its CSR program, AVID integrates well in serving a school with a high 
percentage of low income students. When boundary lines were redefined, however, the rating 
slipped. Moreover, the impact of the more rigorous state exams is yet to be observed, especially 
in light of the NCLB’s requirements regarding adequate yearly progress. 

Although the recombination of students from another school serves to occlude any benefits that 
might be seen in increased test scores, the previous rise in performance ratings is one indication 
of program success. Of 31 former AVID students at Dunbar who have recently graduated from 
high school, 30 have enrolled in college. Given the student demographics at Dimbar, this is a 
noteworthy accomplishment, even when intervening high school years are taken into account. 
Examination of Texas’ Academic Excellence Indicator System reports revealed a pattern, as was 
expected, of increasing performance on state exams and a lowering of the dropout rate imtil the 
merging of the schools. At that time performance declined, including lower attendance rates. 
This confoimd makes it impossible to independently assess the impact of the AVID program 
across time at Dimbar. 
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Houston School 

Plainview Independent School District, Plainview, Texas 
Initial funding and program implementation 

Houston School, located in a recently remodeled former shopping center, serves students in two 
distinct types of programs: an alternative education program (AEP) — a school of choice — and in 
a discipline alternative education program (DAEP). Middle and high school students attend class 
in separate areas of the facility. The high school AEP, which serves both as a credit recovery and 
dropout recovery facility, operates with two four-hour sessions; students may attend either or 
both. Although fewer than half of students are classified as economically disadvantaged, 
Houston’s mobility rate exceeds 75 percent — far above the state average. 

Prior to the CSR grant, students needing credit recovery or otherwise requiring an flexible 
environment (e.g., pregnant or parenting students) were transported to a school in a neighboring 
district, where they worked out of “packets” prepared by regular high school teachers. This 
approach contributed to high dropout and low attendance rates. The need for an alternative 
program located in the district was the impetus for pursuing the CSR grant. The Texas Education 
Agency provided support in submitting the proposal. Houston School received $162,000 
annually over three years of CSR funding. These funds were used primarily to purchase 150 
networked computers, for PLATO education software, and for staff training. While the principal 
has been primarily responsible for determining how CSR grant funds were spent, he has involved 
the counselor, who was instrumental in developing the grant, and instructional staff members in 
making these decisions. 

The diverse needs of AEP and DAEP students, as well as their grade level range, necessitated a 
computer-driven instructional approach. After visiting other schools using PLATO, the principal 
and counselor determined that it would meet their students’ needs and cost less than other 
systems. PLATO consists of self-paced instruction delivered primarily through interactive 
computer programs. A large number of subject areas and grade levels are available. Support 
provided by PLATO was very good, and tec hn ical problems were quickly addressed. All 
teachers at the school have received training in the use of PLATO, and one teacher was primarily 
responsible for the maintenance and operations of the system. 

The post-CSR program 

After three years, PLATO had become institutionalized; no consideration was given to 
discontinuing the program when the grant ended. All Houston School teachers have been trained 
in PLATO and incorporate the lessons into their instruction as needed. The only change was that 
interaction with the PLATO model provider is now restricted to issues regarding problems 
(which are reported to be few). At this time, local funds are used to support the AEP’s PLATO 
program. 

Many students now complete school who previously would not have graduated. With a decrease 
in dropout rates primarily attributed to this school, the program fits well with the district’s goals. 
However, issues surrounding continued use of alternative accountability ratings (not supported 
by NCLB) may eventually present problems. The annual dropout rate performance (4.5 percent 
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for all students in 2001) is very reasonable for an alternative school, yet the same figures, under 
the standard system, could earn the school a “low performing” rating. 

Graduation rates have improved, attendance is up, and fewer students drop out. Houston’s 
teachers and coimselors also said that the students’ sense of pride had increased tremendously. 
Few students want to return to the regular high school. One student said that she would never 
have finished there, but was now graduating with honors and had been accepted into college. 
With the increase in graduation rate, the principal said that teachers felt better about being at the 
school, and were less likely to leave than previously. A sense of accomplishment permeates the 
school. There is typically a waiting list to attend the school. 

As might be expected, parental involvement was not high at Houston School, especially for the 
high school. Some of these students are “on their own” or have less than positive home 
situations. Also lacking was a strong, formal evaluation component. The principal interviewed 
each graduate, but the information does not appear to have been incorporated in any formal 
manner into modifying the program. 
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Irving Middle School 

San Antonio Independent School District, San Antonio, Texas 
Initial funding and program implementation 

Irving Middle School currently serves 620 students in grades 6-8 in central urban San Antonio, 
where families are predominantly low income Hispanic students. Participation in CSR was a 
leadership initiative by a new superintendent facing the highest number of low-performing 
schools in the state and motivated to make changes. Previous school improvement efforts were 
described as unsuccessful “band-aid” approaches. The district encouraged a first cohort of 16 
schools to apply for CSR grants. The original grants focused on professional development 
initiatives, and funds were funneled from national entitlements through the Texas Education 
Agency and disbursed by the district. Once the initiative was established, the New American 
Schools Design (NASD) institute was invited to come to the district to present a variety of 
models. The superintendent, who was familiar with NASD educational research, approved the 
models offered and provided leadership in the selection process. Irving’s focus has been 
technology driven since the 90’s, and thus the Co-Nect model was chosen to support this drive. 

The CSR coordinator made spending decisions. Grant funds, supplemented by district Title I 
money, initially paid for computers, professional development for teachers, and classroom 
library materials. When the program changed in the second year to Problem Based Learning 
(PBL), CSR paid for training, computer hardware, and technical services. Other entitlement 
funds paid for professional development and class size reduction. 

Although training in PBL techniques provided by the regional educational service center was 
considered good, teachers needed more time to absorb it. Eventually about half of the teachers 
were trained. The relationship with the provider was not sustainable without funding. 

The program post-CSR 

Interviewees agreed that the lowest achieving students were empowered as a result of CSR, and 
that their performance has improved since its implementation: “average” to “below average” 
students’ motivation was most impacted because they experienced success for the first time and 
began to view learning as much more than “passing tests.” 

PBL reorganizes the classroom and demands preplanning. Thus participating in CSR shifted the 
teaching role away from textbooks and toward a learning facilitator role. Training helped 
teachers with accountability for learning. A teacher said that training transformed her life and 
that PBL should be for all students. Participation also upgraded the technological skills of 
teachers and students. Still, many teachers were resistant to the change. 

The organizational focus provided by CSR has transformed Irving’s culture and given the district 
a sense of confidence that every school can attain “exemplary” status in the near future. Despite 
this success, however, schoolwide, student-centered PBL efforts have diminished, and the 
practice is now used by only a few teachers. The new state assessment system is rigorous, and 
teachers hesitate to depart from its scope and sequence. On the other hand, technology in place as 
a result of the CSR grant continues to be a strong element in the school’s identity. 
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Gilcrease Intermediate School 
Tulsa School District, Tulsa, Oklahoma 

Initial funding and program implementation 

Gilcrease Intermediate School serves 450 students in grades 5 and 6. Students are predominantly 
African American and receive free/reduced price limch. Before CSR, Gilcrease had a “Read 
180” program that served 10 percent of its students. When the state department of education 
contacted the principal about the CSR grant opportunity, he was interested because the grant 
would increase student support with minimum district involvement. Based upon research 
findings showing effective educational programs for students similar to its population, Gilcrease 
initially opted for three models: Integrated Teaching Instruction, Great Expectations, and 
“resiliency strategies,” which uses reading coaches and home visits. During the first year, a local 
university provided technical assistance. Expensive first year training required more fimds than 
expected. 

The second year, because student performance was not immediately impacted, Gilcrease chose to 
substitute Success for All (SFA) for the other models. Part of the appeal was the way SFA’s 
structure provides a consistent curriculum. SFA training was comprehensive; a Tulsa consultant 
facilitated monthly trainings at Gilcrease. 

The program post-CSR 

SFA supports NCLB goals; however, standardized assessments such as ITBS and SAT 9 
measure outcomes different from those of the CSR initiative programs. Gilcrease’s principal said 
the district is going in a “different direction” from that of CSR. High student mobility — 60 to 70 
percent — ^has prompted a district effort to put schools on “the same page,” to increase 
educational continuity. Thus the district has adopted a model of reform, the Tulsa Reads 
Program. Gilcrease’s SFA program is no longer fimded and has been discontinued. Similarly, at 
the nearby elementary school where SFA had been implemented for three years through CSR, 
the district considered the student progress under SFA insufficient and has since adopted the 
NCLB Reading First strategy. After their CSR grant period ends, leftover SFA materials will be 
incorporated into the general curriculum, but the program will not be continued. Interviewees 
would have preferred to stay with SFA because they found its characteristic progression to be 
engaging for students; however, now that the new schoolwide educational initiatives have 
impacted the school culture, there is “no way back.” 

Both Gilcrease and the elementary principal agreed that the CSR program’s most important 
impact was its systemic change. Prior to SFA implementation, faculty resisted schoolwide 
programs; now most teachers support schoolwide practices. The elementary principal noted that, 
to maintain consistency, annual trainings had been necessary. To increase the program’s 
effectiveness, he suggested that its structure should deal more effectively with teacher turnover. 

With SFA, all students displayed great progress in the classroom, and “at-risk” students were 
energized and empowered by their learning experiences. SFA students were grouped into reading 
levels and progressed consistently. Teachers utilized classroom reading achievements as a visible 
and measurable growth tool for moving students to appropriate reading levels. Students were 
motivated by their sense of success and progression; attendance was positively impacted. 
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Ysleta Elementary School 

Ysleta Independent School District, El Paso, Texas 
Initial funding and program implementation 

Ysleta Elementary School serves 620 students in grades K-6 — nearly all Hispanics from low 
SES families; more than half, bilingual. Although the school has a large migrant population, it 
has earned the “Recognized” rating for academic performance. 

When the principal learned about the CSR funding opportunity at a district leadership institute, 
the school was aheady participating, with district support, in the El Paso Collaborative for 
Academic Excellence (“the collaborative”), a partnership that included the Industrial Areas 
Foundations (lAF) and the Washington, DC based Center for Planning and Economic Policy 
(CPEP). The University of Texas at El Paso provides technical support. At that time, the school 
also had training grants and parent academies. Still, prior to CSR funding, there had been no 
effort toward schoolwide reform. 

In 1998, the school received $50,000 for each of three years. The site-based management team 
worked with the collaborative to enhance the existing program and made decisions on expending 
CSR funds, which were used to furnish classrooms and to bring together culture and Reading 
Recovery approaches with a single focus on students’ success. Previously, the collaborative’s 
involvement at Ysleta Elementary had been paid by grants from the Texas Education Agency, by 
investment capital fund (IFC) and improving teaching and learning (ITL) grants, and by Alliance 
Schools challenge grants. CSR funds now paid for consultants who worked with these grants, for 
resources needed to build up classroom libraries, and for teacher training. 

The program post-CSR 

After CSR funding ended, continuation of activities was supported by local Title I funds and by 
state funds from ICF and ITL grants. The principal also recaptured and redirected funds. 
Members of the collaborative continue to meet and plan. CPEP provides the school with a 
consultant at no cost. Despite this support, more funds will be needed for ongoing training, 
staffing, and materials. Student success initiatives for reading academies and for math and 
science have been funded through Title I and II allocations, and Title III has served to support 
the program for bilingual education. However, there is concern over the potential issue of losing 
state funding, which would make it difficult to continue funding these important components of 
the school’s curriculum. 

Ysleta Elementary’s program was designed by teachers and students. Its goals align well the 
Texas accountability system and with district, state, and federal goals. Although not yet reflected 
on state exams, the principal reported a transformation of teachers, students, and parents, who 
now “learn as a community.” All have benefited. The program’s approach is well suited for the 
bilingual education needs of students, and the 500 laptop computers donated to the school, 
networked with Ethernet, have benefited the at-risk students. Students see themselves as capable 
readers and are engaged in life-long learning habits. The parental involvement component has 
positively affected difficult-to-reach students and children of migrant workers. Parents are 
increasingly willing to spend money on books. Teachers now conduct academies for parents to 
learn to teach students at home — a dramatic cultural shift for these families. The coordinator said 
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that the classroom impact was “tremendous.” Classroom libraries — ^previously limited — ^now 
include Spanish language books. Student performance is monitored constantly through pre- and 
post-testing observational reading components for primary grades and developmental reading 
assessment for all students. Teachers also have integrated student-directed projects into portfolio 
assessment. 

Training, provided by the collaborative — initially at the central office and afterwards through 
summer institutes — is regarded as the most important and transformative factor in the program’s 
success. Weekly meetings now provide ongoing learning, as teachers train their peers. As 
teachers continue to test the model in the classroom, the collaborative refines and integrates best 
practices into the program. Teachers have developed expertise in reading, in balanced literacy 
with bicultural awareness, and in language arts and literature, and are now considered adjunct 
professors at New Mexico State University. The school provides model teaching instruction for 
graduate students. The impact of this collaboration has spiraled from an isolated school’s success 
to a model of teaching that addresses adjustments to the special needs of the population it serves. 
This program gains from its success, as it has obtained support from leadership within the 
academic and educational community. Possibly because of the lAF’s role, this program also 
enjoys the support of residents and businesses of the city. Finally, the school’s transformation 
has had a positive impact for all district schools that have embraced the collaborative model. 

Nonetheless, there is fiiistration over the lack of continuity in policy mandates: “Every year is 
different.” Variables beyond the school’s control — lack of funding, teacher shortages, the 
superintendent’s visions and directives, and federal and state mandates (e.g., NCLB and “the 
tests”) — constantly seem to contradict previous initiatives. The state-mandated test is a particular 
challenge because it does not take into account a population’s demographics (i.e., migrant 
students often bring very little knowledge). A particular challenge for the school has been the 
high teacher turnover and the high level of student mobility. Meeting this challenge requires 
cultural change. The situation has stabilized as students have experienced more success, and as 
parents have learned to try to keep their children in this school. 
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Survey A: Schools should complete this survey if all or portions of the Comprehensive School Reform 
Demonstration (CSRD) program ARE STILL IN PLACE in 2002-03 



Please circle or check the best response. You may attach additional sheets for explanation as needed. 
Please leave response blank if you have no opinion or question is not applicable. 

\. What is your position? Principal Assistant Principal Teacher Program Coordinator Other 

2. How many years have you been in this school? 

3. Is the person who originally put this CSRD program in place still employed in this school or in this district? 

Yes ^No Don’t Know 



4. Is the CSRD program originally supported with federal funds currently being implemented in this school? 

^The entire CSRD program is currently in place in this school. 

^About half of the CSRD program is currently in place in this school. 

Only a small portion of the CSRD program is currently in place in this school. 

5. Please check the primary reasons why at least some aspects of the CSRD program are still implemented in this school. Check all 
that apply. 

CSRD program achieved successful student performance outcomes CSRD has strong support by teachers or administrators. 

Funding for program was provided by local (or other) funds. CSRD program has strong parental support. 

CSRD program is aligned with school goals. Other (specify) 

Of these, which is the most important reason for program continuation. Please explain. 



6. CSRD programs include nine comprehensive reform components. To what extent does your program implement the following CSRD 
components? Circle the best response. Please refer to a description of each component attached to this survey. 



' ■ ■■ ■ '"'is- 

»■■ 4 , ■; - 

" • ■ . . i - I 

CSRD Components 

..v>- ■■>>?: >; ■■ ■;■■ -'4. j 

.. ■ .. 


To what exteridCdid this CSRD program 
implement the following components? 


Is this component still in 
place as it was] originally 
implemented? ] 


Was this component ^ 
effective iii prbdiicihgf 
the desired results? ■ 

V > .. j; ^ ^ 


' " Fully"''’ 

Implemented 


Somewhat 

implement^ 


NeVeir 

implemented 


-- Yes 


e .. No., . 


* - 

Yes i 




Don’t 

Knbw 


Use effective, research- 
based methods 


1 


2 


3 


1 


2 


1 


2 


3 


Use a comprehensive 
design with aligned 
components 


1 


2 


3 


1 


2 


1 


2 


3 


Provide professional 
development 


1 


2 


3 


1 


2 


1 


2 


3 


Develop measurable 
goals and benchmarks 


1 


2 


3 


1 


2 


1 


2 


3 


Have support within the 
school 


1 


2 


3 


1 


2 


1 


2 


3 


Have parental and 
community involvement 


1 


2 


3 


1 


2 


1 


2 


3 


Use external technical 
support and assistance 


1 


2 


3 


1 


2 


1 


2 


3 


Evaluation strategies 


1 


2 


3 


1 


2 


1 


2 


3 


Coordination of 
resources 


1 


2 


3 


1 


2 


1 


2 


3 



Please Return Survey by April 17, 2003 
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7. Which of the following three stages of school improvement based on the CSRD model selected did your program achieve? 



(See back of letter) 



Stages of School Improvement 


Was this stage completed? 


In what year was this stage completed? 


Initiation/Mobilization 


Yes 


No 




Implementation 


Yes 


No 




Institutionalization 


Yes 


No 





8. Please tell us how this CSRD program has changed now that CSRD funds are lo longer available. 



9. We are interested in your opinion regarding the CSRD program in your school. Please indicate the best response. 





Yes 


No 


Don’t 

Know 


Do you think this program should be implemented in other similar schools? 


Y 


N 


DK 


Has this program spread to other schools in the district or schools in nearby districts? 


Y 


N 


DK 


Would you implement this program if you went to another school? 


Y 


N 


DK 


In retrospect, do you this CSRD model worked well for this school? 


Y 


N 


DK 


Do you still use the CSRD vendor for support? 


Y 


N 


DK 


Do you think this program will continue to be implemented in this school? 


Y 


N 


DK 


Do you believe that having a CSRD grant has a positive effect on: 
Student achievement? 


Y 


N 


DK 


Parental Involvement 


Y 


N 


DK 


Professional Development 


Y 


N 


DK 


School wide reform? 


Y 


N 


DK 



10. Please rate quality of model used for the CSRD Poor Fair Good Excellent 

1 1 . Please rate quality of support received from the model provider. Poor Fair Good Excellent 

12. Please rate, in order of importance (1-7,) which of these outcome measures are the best way to assess the effectiveness of the 
CSRD program on this school 

Student test scores Student grades School completion 

Student portfolio Parental involvement Student engagement 

Student attendance 

13. Are there any other outcome measures you would use to assess the effectiveness of this program? Please list: 



14. Please describe the most important changes in your school that resulted from the CSRD program. For example, did or has the 
program impacted school organizational changes, reallocation of resources, staffing, scheduling, or student/teacher 

ratios. 

15. Did the program spend all of the money awarded to implement the CSRD program ? Yes No Don 7 Know 

If NO, why were the funds not spent? 



16. How is the CSRD program currently funded? 



What percentage of these funds comes from the following sources? Local funds Vo Federal Funds Vo Other funds % 

17. Was the CSRD program integrated into the school’s School Improvement Plan and all other efforts toward 
continuous 

improvement? 



Please Return Survey by April 17, 2003 

er|c 
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Survey B: Schools should complete this survey if the Comprehensive School reform Demonstration 
(CSRD) program IS NO LONGER IN PLACE in 2002-03 



Please circle or check the best response. You may attach additional sheets for explanation as needed. 
Please leave response blank if you have no opinion or question is not applicable. 

18. What is your position? ^Principal Assistant Principal Teacher Program Coordinator Other 



1 9. How many years have you been in this school? 

20. Are the person(s) who originally put this CSRD program in place still employed in this school or in this district? 

Yes No Don’t Know 



2 1 . How many years was the CSRD program in place in this school? years Don V Know 

22. What year was the CSRD program discontinued? Don V Know 

23. Please check the primary reasons why the CSRD program was discontinued in this school Check all that apply, 

I don’t know why the program was discontinued 

Did not achieve successful student performance outcomes Did not have support by teachers and/or administrator 

Did not obtain local (or other) funding to continue program Did not have strong parental support for program 

Did not align with school goals CSRD Program was too expensive to continue 

^Did not obtain adequate or quality support from model provider ^Chose a new model 

^CSRD Program was no longer needed Did not have adequate leadership 

Other {specify) 

Of these, which is the most important reason that prevented the program from continuing? Why? 



24. What would have made a difference so that the CSRD program would have continued in this school? Can you list any 

problems that could have been fixed? 



25. CSRD programs include nine comprehensive reform components. To what extent did the program implemented in this school 
use the following CSRD components? Circle the best response. Please refer to a description of each component attached to this 
survey.. 



. . T- -v- -i- •$- f' j' '7^ 

' T .'i: Syr W "■ " 

' X. r is- js, . *>■ 

Did the CSRD program this 


■ ■■ ■■■': r 

To what extend^did this CSRD program ? > ? 
implement the following components by the ^ 
endof.2tfbl.f! t ! ! " ‘ \ 


4 f Was this component 
" effective in producing the 
7 f desired results? 

4 's !' ■>?{■ ■■ 5 


school ,, X,. o' 

" '■ -- ^ ^ 4 -’I- 

4-4 .,xv '4'- .5 ■ 


/■ . 

Implemented^ 


4 .r«. 'vr 4 >4 

f;' Somewhat- 
, Implemented ^ 

4 ■ -V.. \ f. J. 


■ ^ .So 

- Never’ " ;> 
Ii^lementedt 


mi 

j ^ 


J No; : 

H \ ■ 


4 " ■ 

^ Don’t 
^ Know : 


Use of effective, research-based methods 


1 


2 


3 


1 


2 


3 


Use a comprehensive design with aligned 
components 


1 


2 


3 


1 


2 


3 


Provide professional development 


1 


2 


3 


1 


2 


3 


Develop measurable goals and 
benchmarks 


1 


2 


3 


1 


2 


3 


Have support within the school 


1 


2 


3 


1 


2 


3 


Have parental and community 
involvement 


1 


2 


3 


1 


2 


3 


Use external technical support and 
assistance 


1 


2 


3 


1 


2 


3 


Evaluation strategies 


1 


2 


3 


1 


2 


3 
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Coordination of resources 



1 


2 


3 |__! 


2 


3 



26. We are interested in your opinion regarding the CSRD program in your school Please indicate the best response. 



-if •- 'X ■' ^ ’ "i r ^ V '.I ."V _ ^ 

.s'v ,j! ' ■ ■’7 ..I,.. .’’v •' . ■ '■ 


Yes 


-.No- 


^ ’ 4Khow : i 


Was the CSRD program successful in this school? 


Y 


N 


DK 


Do you think this program should be implemented in other similar schools? 


Y 


N 


DK 


Has this program been implemented in other schools in the district or schools in nearby districts? 


Y 


N 


DK 


Would you implement this program if you went to another school? 


Y 


N 


DK 


In retrospect, do you think the model selected for this CSRD program was the best model for this 


Y 


N 


DK 


school? 








Would you implement program if funding were available? 


Y 


N 


DK 


Do you believe that having a CSRD grant has a positive effect on: 








Student achievement? 


Y 


N 


DK 


Parental Involvement 


Y 


N 


DK 


Professional Development 


Y 


N 


DK 


School wide reform? 


Y 


N 


DK 


Did you use the CSRD vendor for support? 


Y 


N 


DK 


If yes, was the vendor or model provider support effective? 


Y 


N 


DK 



27. Please rate quality of model used for the CSRD program. Poor Fair Good Excellent 

28. Please rate quality of support received from the model provider. Poor Fair Good Excellent 

29. Please rate, in order of importance (1-7,) which of these outcome measures are the best way to assess the effectiveness of the 
CSRD program on this school 

Test scores Grades School Completion 

Portfolio Parental Involvement Student engagement 

Attendance 

Are there any other outcome measures you would use to assess the effectiveness of this program? Please 
list: 



30. Please describe the most important changes in your school that resulted from the CSRD program. For example, did or has the 
program impacted school organizational changes, reallocation of resources, staffing, scheduling, or student/teacher ratios. 



3 1 . Did the program spend all of the money awarded to implement the CSRD program ? Yes No Don 't Know 

If NO, why were the funds not spent? 



32. Which of the following three stages of school improvement based on the CSRD model selected did your program achieve? (See 



back of letter) 



Stages of School Improvement 


Was this stage completed? 


In what year was this stage completed? 


Initiation/Mobilization 


Yes 


No 




Implementation 


Yes 


No 




Institutionalization 


Yes 


No 





Was the CSRD program integrated into the school* s School Improvement Plan and all other efforts toward continuous improvement? 
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Site Visit Interview and Write-Up Guide 



Name of Interviewee 

Position 

Contact information 



CSRD District Coordinator/ School Principal/ CSRD Campus Coordinator 

This interview guide will be used for the school principal, the district CSRD contact, and the CSRD 
coordinator at the campus level. Not all questions are appropriate for each interviewee. Not all sites will 
have a unique person for each of these roles. 

Thank you for participating in the evaluation study of the CSRD program conducted by. 
Southwest Educational Development Laboratory (SEDL.) Academic Information Management, 
Inc. (AIM), a contractor, is assisting SEDL in conducting this study. 

The primary purpose of this study is to identify the status of reform programs that received 
CSRD funds from 1998-2001. We specifically want to know the nature and status of the 
program, how it is working and how is it funded. How has it changed after CSRD funds were no 
longer available? 

1. Please tell us about yourself. As a (PRINCIPAL/ADMINISTRATOR/COORDINATOR, 
ETC.) what is/was your role regarding the CSRD program? 

2. How long have your been involved with the program? 



Program Implementation/Funding 

3. Who makes decisions about funding for the program (formerly funded by CSRD)? Is 
this the same person that made decisions when the program was funded by CSRD? 



4. Can you tell me how much additional funding was requested for the program when the 
program was funded by CSRD? How much supplemental funding did you obtain for the 
program? What was the source for supplemental funding? 



5. How did the expenditure of funds change over the course of the program (e.g., was more 
money spent on training in the first year, equipment in the second, etc.)? Also, have the 
categories of expenditures shifted now that CSRD funds are no longer available for this 
program? 



6. Did you already have a similar program prior to CSRD funding? 




88 



95 



7. What roles did the district and school play in the development of the funding 
applications? Did your school initiate the request to apply for funding or was this a 
district-level initiative? 

8. Why and how did you select the CSRD model that was implemented in this school? Who 
was involved at the schools in the model selection process? Which criteria were used? 

9. How did you obtain information on the models? What district level assistance, if any, did 
you obtain? 



Post funding questions 

10. Why did you decide to continue funding the program after CSRD funding ended? List the 
specific reasons why you chose to continue the program in this school. 

11. How does the CSRD align with school performance goals? 

12. How did you secure continued funding for the program? Did you apply for a new grant? 
Who was involved in this process? Was this a district or school level decision? Please 
indicate all current funding sources for the continuation of the CSRD program. 

13. How do you coordinate funding for the CSRD program with other models or reform 
efforts taking place on this school? Describe the programs link to school/district goals, 
state goals, and national goals (i.e. No Child Left Behind.) 

14. Are you a member of any consortium with other schools or districts to apply for funds to 
continue this program? If yes, how did you become part of this group and how is it 
working? 
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15. How is the current program funded? Explain which of the following expenditures are 



Type of Expenditure 


Federal 


State 


Local 


Other 


Administrative costs 










Instruction 










Staff Development 










Materials 










Capital Outlay/Equipment 










Other 











16. Were there any unexpected costs directly attributable to the CSRD program? For 
example, additional materials, more in-depth training, etc. If possible, estimate the 
additional costs for each of these unexpected areas. 

17. How has the program changed since funding ended? What aspects of the program are 
intact and what aspects were deleted or modified? Who is responsible for making these 
decisions? 

18. With current funding, do you think the program is as successful as it was when it was 
supported by CSRD? What student outcomes have suffered? Why? 

19. Did you have enough resources to continue to implement the program? Please explain. 

20. For how long is the program funded? Do you think that the program will be in place for 
several years? Is it currently at risk of being dropped? 



Overall program effectiveness/impact 

21. What is your overall impression of the CSRD program? What worked? What did not? 
What needs to be changed? 

22. What kinds of changes, if any, do you see taking place in this school as a result of CSRD 
program? 

23. What students does the program most positively impact? Can you give some examples? 

24. What do you consider to be the one best outcome of the CSRD program? 



25. What outcomes measures are available to measure the success of the program? These 
data will be requested by AIM towards the end of the month. Will you be able to provide 
this data? 

26. How do staff members perceived the program? What do they like? What do they dislike? 

27. Did you contract with a CSRD model provider or someone associated with the model for 
training? What is your overall impression regarding the model providers support and 
training? Can you tell me what worked and what did not? 

28. Have you continued your relationship with the model provider? Do they still provide 
training? How has the training changed over the years of the program? 

29. What level or amount of training is required to maintain the CSRD program? Are the 
training needs different now than they were when the program was first implemented? 

Are you able to meet the current training needs to effectively implement the program? 

30. What measures can be taken to improve the effectiveness of the program? Can this be 
achieved with current funding levels? 

31. There are nine CSRD implementation components. What do these components mean to 
you? How are these components implemented in this school? Can you give an example of 
how these are still being implemented in the CSRD program? 



Use effective, research-based methods: 



Use a comprehensive design with aligned components: 

Provide professional development: 

Develop measurable goals and benchmarks: 

Have support within the school: 

Have parental and community involvement: 

Use external technical support and assistance: 
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Evaluation strategies: 



Coordination of resources: 



Any other comments? 



CSRD Teacher (at some sites, a small group of teachers may he appropriate) 

1. Please describe your role regarding the CSRD program. How long have you been part of 
the program? 

2. Briefly describe how the CSRD program has impacted your classroom. 

3. How does the CSRD align with school performance goals? 

4. Have you and other teachers involved in the CSRD program received adequate training? 

5. What is your overall impression regarding the model providers support and training? 

6. What is your overall impression of the CSRD program? What worked? What did not? 
What needs to be changed? 

7. What kinds of changes, if any, do you see taking place in this school as a result of CSRD 



8. What students are the most positively impacted by the program? Can you give some 
examples? 

9. What do you consider to be the one best outcome of the CSRD program? 

10. How do staff members perceive the program? What do they like? What do they dislike? 

1 1. Do you feel the program is adequately funded? Do you get the support that you need to 
effectively implement the program on this school? Do you get district-level support? 

12. How long do you think this school will keep this program? Why? 



13. Is there anything else that you would like to share with us regarding this program, its 
funding or its effectiveness? 



program? 
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School Counselor 



1 . Please describe your role regarding the CSRD program. How long have you been part of 
the program? 



2. Briefly describe how the CSRD program has impacted students. Which students were 
helped the most? 

3. How does the CSRD align with school performance goals? 

4. What is your overall impression regarding the model providers support and training? 

5. What is your overall impression of the CSRD program? What worked? What did not? 
What needs to be changed? 

6. What kinds of changes, if any, do you see taking place in this school as a result of CSRD 
program? 

7. What do you consider to be the one best outcome of the CSRDiprogram? ’ • • i v 

8. How do staff members perceive the program? What do they like? What do they dislike? 



9. Do you feel the program is adequately funded? Do you get the support that you need to 
effectively implement the program on this school? Do you get district-level support? 



10. For how long do you think this school will keep this program? Why? 



1 1 . Is there anything else that you would like to share with us regarding this program, its 
funding or its effectiveness? 
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